Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Student Guide
Education Services
July 2013
Course Introduction
Course Introduction
Upon completion of this course, you should be able to configure FC and iSCSI SANs in regular
and interoperable fabrics. You will use SAN tools and perform advanced configurations such
as NPIV and troubleshooting.
Course Introduction
Course Introduction
This module focuses on the tools used for managing Connectrix switches, as well as hosts,
initiators and their disk partitions. The module gives an overview of different storage arrays
and Connectrix products.
There are many types of Host Adapters. In this course, we discuss the following three types.
Host bus adapters are hardware components installed in an open systems host to access
storage in a SAN. These drivers are responsible for encapsulating and un-encapsulating SCSI3 protocol (commands and data) within the payload of Fibre Channel frames. In several
environments, hosts also use HBAs to boot the OS from a SAN-attached storage array. In this
situation, the HBA has a special boot code on it to allow the host to probe for SCSI disks
during boot time.
iSCSI HBAs are ideal initiators for iSCSI connections to storage. The iSCSI HBA offloads TCP/IP
and iSCSI frames, reducing the strain from the hosts CPU. iSCSI can be accomplished through
a regular NIC card by installing a software iSCSI initiator in the operating system.
Converged Network Adapters are intelligent multi-protocol adapters that provide host LAN
and Fibre Channel SAN connectivity over 10Gbps Ethernet using Fibre Channel over Ethernet
(FCoE) and Enhanced Ethernet functionality. Unrivaled scalability and industry-leading
virtualization support. Full hardware offload for FCoE protocol processing reduces system
CPU utilization for I/O operations, which leads to faster application performance and higher
levels of consolidation in virtualized systems. Cutting edge 10Gbps bandwidth eliminates
performance bottlenecks in the I/O path with a 10X data rate improvement versus existing
1Gbps Ethernet solutions.
In a UNIX environment, a physical storage device (disk drive) attached to the system is
accessed via a device special file. A device special file is created by the operating system to
provide I/O access to the physical devices. Device special files are like ordinary files in that
they have names, appear in the directory structure, have access protection and can be used
in almost every way that ordinary files can be used.
In the early days of UNIX, the entire drive had to be allocated to the operating system or an
application (the entire drive was allocated to a file system if a file system was needed). The
obvious disadvantage was the lack of flexibility. If a file system ran out of space, there was no
easy way to extend the size of the file system.
The naming convention for device special files usually looks like: c#t#d# , where, c# is the
SCSI controller number, t# is the target ID of the SCSI device and d# is Logical Unit Number
(LUN) of the SCSI device *(e.g. c1t1d0 = Controller 1, Target ID 1 and
LUN 0.
With the introduction of partitioning, the naming of the device special files includes the slice
or partition number as well e.g. c1t1d0s3 Slice 3 on the device
c1t1d0).
The s2 partition is reserved for, and refers to, the entire disk. Thus, c1t1d0s2 allows access to
the contents of the entire physical device.
LVMs introduce a logical layer between the operating system and the physical storage. LVMs
have the ability to define logical storage structures that can span multiple physical devices.
The logical storage structures appear contiguous to the OS and applications.
The fact that logical storage structures can span multiple physical devices provides flexibility
and additional functionality:
In LVM terminology, each physical disk device connected to the host system is a physical
volume (PV).
For each physical device, two device special files are created by the operating system. A
character device and a block device. A character device is also called a raw device. I/O to raw
devices is done on a character level, and there is no caching by the operating system. The
response from a raw device is faster than that of a block device. Raw or Character devices
are typically used for paging, or dump spaces, by the operating system. Databases can also
use raw devices directly, and would manage how the data is stored on the device. Block
devices are typically used by file systems which store data in a structured, hierarchical
manner.
In a Symmetrix environment, the SLVs are viewed like any other physical device. The host
and most LVMs are unaware of the fact that the device is actually RAID-1 or RAID-S protected
or that the device has SRDF or BCV attributes.
LVMs convert the physical storage provided by the physical volumes to a logical view of
storage which is then used by the operating system and applications. One or more physical
volumes are grouped to form volume groups and then Logical Volumes are created within
the volume group. We discuss volume groups and Logical Volumes in the next two slides.
A volume group is created by grouping together one or more physical volumes. A unique physical volume
Identifier(PVID) is assigned to each physical volume, when it is initialized for A volume group is created by
grouping together one or more physical volumes. A unique physical volume Identifier(PVID) is assigned to each
physical volume, when it is initialized for use the LVM. physical volumes can be added or removed from a
volume group dynamically. physical volumes cannot be shared between volume groups, the entire physical
volume becomes part of a volume group.
Each physical volume is partitioned into equal-sized data blocks called Physical Partitions, or Physical Extents,
when the volume group is created. The size of a logical volume is based on a multiple of the Physical Partition.
Logical Volumes are discussed in the next slide. HP-UX calls these equal-sized data blocks Physical Extents while
AIX calls them Physical Partitions. VERITAS uses sub-disks (to be discussed later).
The data that completely defines a volume group is written in the volume group Descriptor Area (VGDA) of the
physical volume. The VGDA on each of the physical volumes within a volume group are identical. HP-UX writes
the volume group information into the volume group Reserved Area (VGRA).
The volume group is handled as a single unit by the LVM. Thus, a volume group as a whole can be activated or
deactivated. Typically a volume group would contain related information, e.g. each host will typically have a
volume group which holds all the OS data (AIX rootvg, HP-UX vg00, VERITAS rootdg). Applications would
be on separate volume groups. At least one disk group is needed for the Operating System. Application and
Operating System data maintained in separate volume groups
Logical Volumes are created within a given volume group. A logical volume can be thought of as a virtual disk
partition, while the volume group itself can be thought of as a disk. A volume group can have a number of
Logical Volumes. We discuss Logical Volumes in the next slide.
Physical partitions:
Default size is 4 MB
PP sizes of 1 - 256MB (in powers of 2) may be specified
AIX 4.2.5 = Max 256MB x 1016 PP
AIX 4.3.x = Max 2048MB x 1016 PP
AIX 5L 5.3 Max 128 GB x 2048 PP (No longer per disk but to entire VG)
Logical volumes are made up of a number of logically contiguous data blocks or logical
partitions. The logical volume appears as a physical device to the Operating System and is
accessed via device special files (character or block). After creating a volume group, Logical
Volumes can be created.
Each logical partition of a logical volume maps to at least one physical partition on a physical
volume within the volume group. Under normal circumstances, there is a one-to-one
mapping between a Logical and Physical Partition. A one-to-many mapping between a
Logical and Physical Partition leads to mirroring of Logical Volumes.
A logical volume can be made up of noncontiguous physical partitions and can span multiple
physical volumes. Logical volumes can be used directly by an application or a file system.
File systems can reside on disks, disk partitions, or on a logical volume created by an LVM. A
file system organizes data in a structured hierarchical manner, via the use of files and
directories. Apart from files and directories, the file system is also made up of a number of
other structures, which are collectively called the Meta Data. The Meta Data consists of the
Superblock, the Inodes and the list of data blocks free and in use. The Meta Data of a file
system has to be consistent for the file system to be considered healthy.
Superblock Contains important information about the file system: File system type,
creation/modification dates, size/layout of the file system, count of available resources and a
flag indicating the mount status of the file system. The Superblock maintains information on
the: #inodes allocated, in use, and free, #data blocks allocated, in use, and free. (This is set
when the file system is created. The number of Inodes allocated = File System Size divided
by the number of bytes per I-node (NBPI)). Each file or directory needs an inode. New files or
directories cannot be created if there are no free inodes.
Inodes An Inode is associated with every file and directory, and has information about file
length, ownership, access privileges, time of last access/modification, number of links and,
finally, the addresses for finding the location on the physical disk where the actual data is
stored.
The meta data of a File System is typically cached in the Hosts Memory Buffers. Host level
buffering is important to keep in mind in a Symmetrix Environment with SRDF and
TimeFinder. The information in a Hosts memory buffer is not available on the BCVs or the
SRDF target devices until they are flushed down to the standard devices.
10
11
The server views each disk resource to be on a single path. The configuration spreads I/O load across
the paths. Each application is set up with its own storage. The storage is allocated to Channel
Directors or Storage Processors based on expected data requirements of the applications. This setup
is done based on snapshot measurements, guesstimates of average loading and predictive loads.
This diagram depicts a snapshot of the system at a moment in time. The depth of the I/O queues is
very unbalanced. Host applications sitting on top of deep queues are not getting the data they need.
If this was the average loading, the System Administrator would reconfigure the system to balance
the load better. In any system, there will be points in time when the load is unbalanced due to one
application receiving heavy I/O requirements.
In this instance, two of the applications are currently causing high I/O traffic. At this point, two
channels are overloaded (depicted by the red disks on the outside, and the pending request stack)
while two other channels are lightly loaded. In a while, the requests will have been handled and the
system will return to a more balanced load. In the meantime, the applications are being data
starved and the users or applications are experiencing less than optimal performance.
With Multipathing software in the system, applications transparently access multipathing devices
instead of the SD (SCSI driver) devices. Multipathing allocates the requests across the available
channels, reducing bottlenecks and improving performance. This diagram shows a similar snapshot,
with multipathing using multiple channels to minimize the queue depth on all channels.
Since the Channel Directors or Storage Processors are writing to cache and not to disks, any Channel
Director/Storage Processor can handle any request. This allows multipathing to constantly tune the
server to adjust to changing loads from the applications running on the server.
Multipathing improves the performance of the server, enabling it to make better use of the storage.
This results in better application performance and less operational resources spent on the care and
feeding of the system and more (financial) value from your server investment.
Multipathing does not manage the I/O queues; it manages the placement of I/O requests in the
queue.
Copyright 2013 EMC Corporation. All rights reserved
12
Automatic: PowerPath algorithms allow the increase of application I/O rates through
Symmetrix and CLARiiON with automatic data path load balancing allowing for
greatest efficiency and throughput. PowerPaths volume manager capability simplifies
disk administration tasks to further reduce total cost of ownership through high-level
commands that hide storage complexity. It automatically manages workloads and
volume expansion.
Optimized: By leveraging your server, SAN, and storage assets, PowerPath maximizes
your investment by increasing storage utilization.
13
14
This lesson covers an introduction of CLARiiON, VNX, and Symmetrix storage arrays and their
management options.
15
In a SAN environment, a storage device commonly known as Target uses disks and tapes.
Tapes are a popular storage media used for backup because of their relatively low cost.
However, tape has various limitations; data is stored on the tape linearly along the length of
the tape. Search and retrieval of data is done sequentially, invariably taking several seconds
to access the data. As a result, random data access is slow and time consuming. This limits
tapes as a viable option for applications that require real-time, rapid access to data. In a
shared computing environment, data stored on tape cannot be accessed by multiple
applications simultaneously, restricting its use to one application at a time. On a tape drive,
the read/write head touches the tape surface, so the tape degrades or wears out after
repeated use. The storage and retrieval requirements of data from tape and the overhead
associated with managing tape media are significant. Even with all these limitations, tape is
not yet obsolete.
Disk drives are the most popular storage medium used in modern computers for storing and
accessing data for performance-intensive, online applications. Disks support rapid access to
random data locations. This means that data can be written or retrieved quickly for a large
number of simultaneous users or applications. In addition, disks have a large capacity.
16
VNX/CLARiiON arrays meet all the criteria for mid-tier storage arrays.
Performance: Mid-tier storage architectures are ideally suited for applications with
predictable workloads that need fast response times and high sustained throughput.
Availability: For applications that require five nines (99.999%) availability, mid-tier
architectures are designed with redundant components
Flexibility: Mid-Tier storage have the ability to scale for increased capacity. Designed
for multi-protocol connectivity.
17
The CLARiiON CX4 series consists of four models - the CLARiiON CX4-120, CX4-240, CX4-480,
and CX4-960. The model numbers refer to the maximum number of drives supported on
each system.
The CLARiiON CX4 series boasts up to twice the performance of the previous generation and
provides up to 2.5 times more processing power. The new CLARiiON CX4 architecture also
delivers twice the capacity scale (up to 960 drives), twice the memory, and twice the LUNs
compared to the previous generation CLARiiON.
With the introduction of CLARiiON CX4, the FLARE operating system has also been upgraded
from a 32-bit to a 64-bit environment. This enhancement enables the scalability
improvements and also provides the foundation for more advanced software functionality in
the future.
All CLARiiON CX4 series models come pre-configured with Fibre Channel and iSCSI
connectivity allowing users to easily add connectivity as the environment grows.
18
19
Navisphere Secure CLI is the preferred command line interface for storage management.
Secure CLI allows the user to perform all the functions needed to manage the array and
includes all classic CLI commands.
Navisphere Classic CLI is the original command line for managing CLARiiON arrays. Classic CLI
can be enabled or disable with the management GUI.
Navisphere GUI is an intuitive, easy-to-use graphical user interface for all management
functions on the array.
Navisphere Service Task Bar (NST) allows the Hardware and Software registration and
configuration of the array.
Navisphere Wizard is a simple, intuitive menu-driven interface which allows a lessexperienced user to manage the CLARiiON.
20
Navisphere Manager provides speed and flexibility using the familiar Microsoft windows
interface. It lowers cost of management/ownership, including training and administrative
costs, and has a proactive focus on addressing potential problems. It reduces personnel
requirements by its ability to increase the productivity of staff by managing larger amounts
of storage with fewer resources.
21
The EMC unified storage systems are grouped into two different series, the VNXe and VNX
series. The VNXe series includes the VNXe3100 and VNXe3300 which is both a File and iSCSI
block solution. The VNX series include the VNX5100 which is FC block only, VNX5300,
VNX5500, VNX5700 and VNX7500. Unified storage platforms combine Block array and File
serving components into a single Unified Block and File, File only, or Block only storage
solution. The VNX series storage systems leverage Intel multi-core CPUs and PCI Express 2.0
interconnects to deliver uncompromising scalability and flexibility while providing market
leading simplicity and efficiency. The VNX series platforms also are designed to comply with
the emerging Energy Star Storage Server power efficiency guidelines.
The VNX series implement a modular architecture concurrently supporting native NAS, iSCSI,
Fibre Channel and FCoE protocols for host connectivity and 6Gb Serial Attached SCSI (SAS)
backend topology. The high end VNX5700 and VNX7500 utilize Storage Processor Enclosure
(SPE) architecture and the mid-range models utilize Disk Processor Enclosure (DPE)
architecture.
22
VNX combines all the protocols that are needed in todays IT environment with simple
unified management features. However, simple to use does not mean simple as the VNX
features advanced replication, management, and Fully Automated Storage Tiering (FAST).
The Architecture is Modular Unified (configured for purpose: File, Block, and Object). The
VNX is designed for high performance which is optimized for Multi-core and Flash and
includes a 6G SAS back end infrastructure. The VNX family is flexible, and by using Expanded
UltraFlex I/O, the VNX can natively support Fibre Channel, iSCSI, CIFS, and NFS. The new
packaging is denser and greener than ever with new Energy Star ratings on energy efficiency.
23
The VNX series unified modular architecture delivers a highly flexible and scalable storage
solution. A VNX7500 can scale up to 60 CPU cores of processing power. There are 12 CPU
cores dedicated to high performance block-serving using six core CPUs on two Storage
Processors. There can be up to 48 CPU cores dedicated to networked File system
management and data sharing via six core CPUs on eight X-Blades. Block connectivity is via
FC, FCoE, and iSCSI, and File connectivity is via NAS including NFS, CIFS, MPFS, pNFS. The
pNFS protocol is only available with VNX arrays.
24
VNX Series systems use X-Blade for File front end and the Storage Processors for block access
to the back end. The data control flow is handled by the Storage Processor in block-only
systems, and the X-Blades in file enabled systems. The Control Station is used to configure,
manage, and upgrade the X-Blades, as well as to manage X-Blade failover.
Each X-Blade Enclosure contains up to 2 X-Blades running VNX OE system software optimized
for file. Depending on the model, a VNX system can contain up to 8 X-Blades. Each X-Blade is
configured with one 4-port 8Gb Fibre Channel I/O module for storage array connectivity and
tape connectivity (for NDMP). Multi blade systems are typically configured with N+1 or N+M
advanced failover (where n is the active X-Blade and M is a pool of standby X-Blades) where
one X-Blade is configured as standby or where a number of X-Blades are configured as a pool
of failover X-Blades for the active blades.
The Disk Processor Enclosure (DPE) or Storage Processor Enclosure (SPE) use dual active
Storage Processors (SPs) for disk I/O. These processors run the VNX OE for Block. The SPE
supports automatic failover should one of the SPs fail.
The disk array enclosures are either 15x3.5disk shelves (Flash, SAS and NL-SAS) or 25x2.5
disk shelves for disk capacity (SAS).
25
VNX hardware and software optimizations enable the VNX to virtualize Exchange, SQL, and
Oracle while providing increased performance for the application. Booting thousands of
desktops and managing the VMware environment are quick and easy. ROI is maximized and
previously missed SLAs can now be met with optimized performance using FAST Cache.
Automatic tiering with FAST VP optimizes disk resources thus reducing TCO. Disk cost can
also be reduced by saving up to 50% disk space with compression and deduplication.
Web/Cloud applications and storage as a service are supported via Atmos VE. Host
encryption, file-level retention, and anti-virus checking provide data security. EMCs proven
replication technologies allow for simple setup, monitoring, notification, and reporting of
local and remote replication. Maintenance, upgrades and troubleshooting are simple and
intuitive with an ecosystem designed for quick answers, software downloads, and problem
resolution.
26
Unisphere is web-based software that allows you to configure, administer, and monitor VNX
series. Unisphere provides the user with an overall view of what is happening in your
environment plus an intuitive and easier way to manage EMC unified storage.
27
There are two ways to use the CLI for the VNX Series Platform:
The Control Station is a customized Linux kernel and operates VNX for file
management services that configure, manage, and monitor Blades. A second Control
Station may also be present in some models for redundancy. If VNX for File or Unified
is present, you can connect to it via serial or SSH to troubleshoot many VNX for File
hardware components.
If VNX for Block is present, the Navisphere Secure CLI can be used. It is a client
application that allows simple operations on the EMC VNX Series platform, and some
other legacy storage systems. It uses the Navisphere 6.X security model, which
includes role-based management auditing of all user change requests, management
data protected with SSL, and centralized user account management.
28
Physical Disks are held in DAEs within the array and are subsequently combined into RAID
Groups. A RAID Group is a set of disks on which you bind one or more logical units (LUNs)
from. A Logical Unit is a portion of a RAID Group that is made available to the client as a
logical disk. Logical Units allow users to subdivide their RAID Groups into convenient sizes for
host usage. With a Traditional LUN, all of the space on it is allocated for usage at the time of
its creation.
29
A RAID group is a set of disks (up to 16 in a group) with the same capacity and redundancy,
on which you create one or more traditional LUNs. A RAID 6 group must have an even
number of disks with a minimum number of four. A RAID 5 group must include at least three
disks. A RAID 3 group must include five or nine disks, and a RAID 1/0 group must have an
even number of disks. The storage-system model determines the number of RAID groups
that it can support.
All the capacity in the group is available to the server. Any RAID Group should consist of all
SAS or all Flash Drives but not a mix of SAS and Flash Drives. Most RAID types can be
expanded with the exception of RAID 1, 3, 6, and Hot spares. Most RAID types can be
defragmented to reclaim gaps in the RAID group with the exception of RAID 6.
30
Storage can consist of two types of storage pools, either Pools and RAID Groups.
A Pool is a collection of disks that are dedicated for use by thin LUNs. A Pool is somewhat
analogous to a RAID group. However, a Pool can contain a few disks or hundreds of disks,
whereas RAID groups are limited to 16 disks.
Pools are simple to create because they require only three user inputs:
Pool Name
Resources (Number of disks)
Protection level: RAID 5 or 6
Pools are more flexible. They can consist of any supported disk drives. Arrays can contain one
or many pools per storage system. The smallest pool size is three drives for RAID 5 and four
drives for RAID 6.
Note: EMC recommends a minimum of five drives for RAID 5 and eight drives for RAID 6.
Pools are also easy to modify. You can expand the pool size by adding drives to the pool and
contract the pool size by removing drives from the pool.
31
High end customers have unique requirements for supporting the information lifecycle
management strategy. These requirements mean that the IT organization must provide
uncompromising levels of service to support the enterprise so it can operate and maintain
the business.
Delivering these capabilities sets Symmetrix Direct Matrix (DMX) apart as the worlds most
trusted storage platform.
32
Companies have unique requirements for supporting the information lifecycle management
strategy. These requirements mean that the IT organization must provide uncompromising
levels of service to support the enterprise so it can operate and maintain the business.
Here are some of the key reasons to choose Symmetrix:
Symmetrix has the unique ability to support multiple tiers within the array, with
software tools to manage more predictable performance and it does massive
consolidation.
Symmetrix provides the lowest total cost of ownership by reducing cost and
delivering higher service levels through scale-out and tiering.
Its the most advanced and widely deployed business continuity platform with unique
capabilities like non-disruptive upgrades, multi-site disaster recovery, and enterprise
consistency.
The Symmetrix family offers the widest choice of solutions in the industry.
33
There are some key Symmetrix V-Max differentiators. For example, Symmetrix V-Max
systems are purpose-built for the virtual environments. High-end capabilities are combined
with unmatched levels of scale, ease of use, and automation to enable virtual environments.
Symmetrix V-Max deliveries higher service levels through scale-out and tiering at the lowest
total cost of ownership. Fully Automated Storage Tiering, or FAST, automatically optimizes
performance to meet service level requirements.
34
Virtual Provisioning was first implemented in Enginuity 5773, and is available on DMX and VMax systems. Virtual Provisioning presents an application with more capacity than is
physically allocated and in some situations may provide a more efficient way of allocating
capacity for applications that are somewhat predictable in capacity growth patterns.
These thin volumes can improve capacity utilization because the actual data is stored a
common pool; only what is used is allocated in the common pool and the pool is shared by
many TDEVs.
In the example illustrated, the host has a 100 GB TDEV, the TDEV uses no disk space, the Thin
pool contains the actual data and only 20 GB is allocated until more space is required. The
allocated capability is managed by EMC software.
35
Symmetrix Management Console is used by Symmetrix for device management for both the
Symmetrix V-Max and Symmetrix DMX products. There are several key features introduced
to simplify storage management in virtual data centers and cluster environments. As data
centers continue to embrace virtualization, management tools are required to tier,
consolidate, and scale physical resources.
Symmetrix Management Console:
Has several ease-of-use functions such as wizards that help streamline the process for
Auto-provisioning, SRDF replication configuration, and enhanced Virtual LUN
technology. Additionally, there is the ability to create storage templates for reuse in
provisioning storage.
Is loaded on the Service Processor, eliminating the need for another server host.
Complements both ProSphere and SYMCLI. It is a lightweight software package with a
web-based GUI.
36
This lesson covered an introduction of CLARiiON, VNX, and Symmetrix storage arrays and
their management options.
37
This lesson covers the different SAN connectivity options and tools used to manage
Connectrix devices.
38
Physically, a Fibre Channel SAN can be implemented using a single Fibre Channel
switch/director, or a network of interconnected Fibre Channel switches and directors. The
HBAs on each host, and the FC ports on each storage array, must be cabled to ports on the FC
switches or directors. Fibre Channel can use either copper or optics as the physical medium
for the interconnect. All modern SAN implementations use fibre optic cables.
Fibre Channel over Ethernet (FCoE) is a new technology protocol, defined by the T11
standards committee. It expands FC into the Ethernet environment. Basically FCoE allows
Fibre Channel frames to be encapsulated within Ethernet frames, providing a transport
protocol more efficient than TCP/IP sharing a single, integrated infrastructure, thereby
reducing network complexities in the data center. FCoE consolidates both SANs and Ethernet
traffic onto one Converged Network Adapter (CNA), eliminating the need for using separate
Host Bus Adapters (HBAs) and Network Interface Cards (NICs). From the connectivity layer
perspective, the use of Fibre Channel Forwarders (FCF) is necessary to service login requests
and provide the FC services typically associated with a FC switch. FCFs may also optionally
de-encapsulate FC frames that are coming from the CNA and going to the SAN and
encapsulating FC frames that are coming from the SAN to the CNA.
An IP SAN solution uses conventional networking gear, such as Gigabit Ethernet (GigE)
switches, host NICs, and network cables. This eliminates the need for special purpose FC
switches, Fibre Channel HBAs, and fibre optic cables. Such a solution becomes possible with
storage arrays that can natively support iSCSI, via GigE ports on their front-end directors
(Symmetrix) or on their SPs (VNX/CLARiiON). For performance reasons, it is typically
recommended that a dedicated LAN be used to isolate storage network traffic from regular,
corporate LAN traffic.
39
The Connectrix family (B-Series, M-Series and MDS-Series) represents the industrys most
extensive selection of networked storage connectivity products. Connectrix integrates highspeed Fibre Channel connectivity, highly resilient switching technology, and options for
intelligent IP storage networking. Connectrix incorporates Fibre Channel or DCB/FCoE
technology for a virtual data center. This wide range of connectivity options allows you to
configure Connectrix directors, switches, and routers to meet any business requirement.
EMCs industry-leading design, implementation, and support services, combine everything in
one complete package.
Connectrix products provide more than just network connectivity. They offer:
Simple, centralized, automated SAN management
Proven interoperability across your networked storage solution
The highest availability to meet escalating business continuity and service level
requirements
Availability to transform your SAN into a virtually limitless and accessible cloud-ready
information resource
Scalability with built-in investment protection
A platform for intelligent fabric-based applications
Connectrix B-series products incorporates 1Gbps to 16Gbps FC speed connectivity and 10GE
DCB/FCoE technology. Connectrix B-series products provide a better way to access, manage,
and protect growing information resources across a consolidated Fibre Channel, Fibre
Channel over Ethernet (FCoE), Fibre Channel over IP (FCIP), Gigabit Ethernet, and optical
network. Some models also serve as a platform for VPLEX, RecoverPoint, and fabric-based
encryption.
Copyright 2013 EMC Corporation. All rights reserved
40
EMC provides a number of management applications and tools to accomplish these tasks.
ProSphere provides centralized management to help streamline SAN management
operations across heterogeneous storage networks. With ProSphere, users can discover the
network topology, view the relationships between components, automate zoning and device
masking tasks, and monitor the SANs health and performance.
In addition to ProSphere, there are individual element managers for configuring, monitoring,
and managing each of the Connectrix products. Connectrix-specific functions such as
installation, firmware downloads, and basic configuration management are performed using
these tools. Element managers include Connectrix Manager Converged Network Edition,
Data Center Network Manager and WEB TOOLS. All element management functions are also
available through a command line interface (through Telnet).
41
42
MDS-Series switches and directors can be managed through CLI or GUI tools. Functions such
as installation, firmware downloads, monitoring and basic configuration management are
performed using these tools.
All element management functions are also available through a command line interface CLI.
The CLI can be accessed from a serial connection through the console port, or through a
telnet or SSH session connected through the mgmt0 (management) port. Not all functions
available through the CLI are available through the GUI.
GUI tools, including Cisco Fabric Manager applications and Cisco Data Center Network
Manager, support Secure Simple Network Management Protocol version 3 (SNMPv3) with a
Java web-based interface.
The Cisco Fabric Manager applications are:
Fabric Manager Client/Server
Device Manager
Fabric Manager Web Services
Performance Manager
The Cisco DCNM-SAN includes these management applications:
DCNM-SAN (client and server)
Device Manager
Performance Manager
DCNM-SAN Web Server
43
This lesson covered the different SAN connectivity options and tools used to manage
Connectrix devices.
44
This module covered the tools used for managing Connectrix switches as well as hosts,
initiators and their disk partitions. Also an overview of different storage arrays and
Connectrix products.
45
46
This module focuses on the tools used to manage B-Series switches and the licensing
schema. Basic switch maintenance and configuration tasks are covered as well.
This lesson covers the B-Series tools for effective SAN management.
The CLI can be used only through a Telnet/ssh Client session in an out-of-band management
environment, using the B-series Ethernet port, or using a console/serial port if available. The
primary purpose of the CLI is to automate management of a large number of
switches/directors with the use of scripts, although CLI commands can also be entered
directly at a command line. Telnet access is enabled by default.
The B-Series CLI is an ideal interface when the following conditions or requirements exist:
At the bottom of the list of available commands are submenus of other commands that can
be run as well (like routing commands under routehelp & Zoning commands under
zonehelp). There are many more commands/actions that are available through telnet than
the GUI (future revisions of the Web Tools GUI should incorporate more of the telnet
functionality).
Note: Before running any switch commands, check the configuration guide to see if there are
any special steps that must be followed.
Web Tools, also called Element Manager, is a graphical user interface (GUI) that enables
administrators to monitor and manage switches and directors from a standard workstation. It
is a factory-installed licensed product that runs on Fabric OS. Web Tools requires any browser
that conforms to HTML version 4.0, JavaScript version 1.0, and Java Plug-in 1.6.0 or higher.
To launch Web Tools, at the browser enter the IP address of the switch as the URL. Enter the
default user name (admin) and password, then click OK. The Web Tools interface opens to
the View window that shows hardware components of the switch.
This slide shows an example of an ED-DCX-4S director and its default view within Web Tools.
For this director, the left side of the window provides activities, monitoring capabilities, and
a fabric view. The center pane displays the physical view of the switch. The right pane
displays switch information and switch events. The action buttons are displayed across the
top of the screen.
Connectrix Manager Converged Network Edition CMCNE is the industry's first unified
network management solution for data, storage, application delivery, wireless, and
converged networks. It supports Fibre Channel SANs, FCoE, IP switching and routing
(including Ethernet fabrics), and MPLS networks providing endtoend visibility across
different network types through a seamless and unified user experience.
CMCNE offers the following capabilities:
Simplifies data center automation through enhanced support for Brocade VCS fabric
technology
This lesson covered the B-Series tools for effective SAN management.
CMCNE is offered with three licensing options; Professional, Enterprise, and Professional Plus
Edition.
CMCNE Professional showcases a subset of features and hardware platforms of CMCNE
Enterprise. The Professional version, which is a no charge version, is intended for the
management of smaller fabrics with Connectrix B-Series departmental switches. It cannot
manage Director class products, has no FICON or routing support, and does not have remote
clients or Call home support.
The Professional Plus is designed for medium-sized businesses or departmental storage
networks. Very similar in functionality to the Enterprise version but limited in
features/scalability by a license key to support up to four fabrics and 2560 switch ports, EDDCX-4S-B directors, encryption products, extension products, and Brocade HBA/CNAs. The
Professional Plus supports both mixed EOS & FOS fabrics and pure EOS fabrics and can be
updated to Enterprise (via a license key update).
CMCNE Enterprise builds on top of the functionality available with CMCNE Professional, and
provides a comprehensive fabric management framework for the end-to-end management
of the data center fabric. CMCNE Enterprise can scale to manage up to 9,000 switch ports
and also contains several new features along with significant enhancements to several of the
existing Connectrix Manager 9.x and Fabric Manager features. Enterprise also supports all
enterprise-class products / technologies including the ED-DCX-B Backbone Director, FICON,
Fibre Channel Routing (FCR), Fibre Channel over IP (FCIP), IP switching and routing (including
Ethernet fabrics), and MPLS networks.
10
In B-Series switches licenses can be associated with a feature version or a blade type. If a
feature has a version-based license, that license is valid only for a particular version of the
feature. If you want a newer version of the feature, you must purchase a new license.
This is an example with a version upgrade. A zoning license that is for Fabric OS version 6.0.0
is added. You can add another zoning license with a version greater than v5.2.0 and above
without removing the zoning license for Fabric OS v5.2.0. Upgrading is allowed but
downgrading is not supported. If a license is not version-based, than it is valid for all versions
of the feature.
If a license is associated with a blade type, the licensed feature can be used only with the
associated blade; if you want to use the feature on a second blade, you must purchase an
additional license.
11
The Connectrix B-Series family offers a number of optional software features that provide
additional capabilities. Here is a brief summary of these optional features.
Inter-Switch Link trunking: Logically merges up to eight ISLs into one, full 8 Gbps
logical link
Advanced Performance Monitor: Provides end-to-end traffic analysis
Fabric Watch: Provides real-time SAN monitoring
Extended Fabrics: Provides greater than 10km of switched fabric connectivity at full
bandwidth over long distances (depending on the platform this can be up to
3000km).
Adaptive Networking: A suite of tools for optimizing fabric behavior and ensuring
ample bandwidth for mission-critical applications; includes Quality of Service, Ingress
Rate Limiting, Traffic Isolation, and Top Talkers
Integrated Routing: Allows ports to be configured as EX_ports supporting Fibre
Channel Routing, eliminating the need to add a PB-48K-18i Fibre Channel routing
blade or use the MP-7800B router for Fibre Channel Routing purposes
Enhanced Group Management: Enables full management of the device in a data
center fabric with the Connectrix Manager Data Center Edition for deeper element
management functionality and greater management task aggregation throughout the
environment
10 Gbps FCIP/Fibre Channel license (10G license): Enables 10 Gbps access on the 16
Gbps FC ports on the Brocade 6510 switch.
12
If you need to install a single or multiple licenses in any B-Series switch, follow these steps:
Connect to the switch and log in using an account assigned to the admin role.
Activate the license using the licenseAdd command.
switch:admin> licenseadd "key
The license key is case sensitive and must be entered exactly as given. The quotation marks
are optional.
For director and enterprise-class platforms, licenses are effective on both CP blades, but are
valid only when the CP blade is inserted into a director that has an appropriate license ID
stored in the WWN card. If a CP is moved from one director to another, the license works in
the new director only if the WWN card is the same in the new director. Otherwise, you must
transfer licenses from the old WWN to the new WWN.
For example, if you swap one CP blade at a time, or replace a single CP blade, then the
existing CP blade (the active CP blade) propagates the licenses to the new CP blade. If you
move a standby CP from one chassis to another, then the active CP will propagate its
configuration (including license keys).
To verify that the license was added correctly use the licenseShow command. The licensed
features currently installed on the switch are listed. If the feature is not listed, enter the
licenseAdd command again.
To remove a license use the licenseRemove command. After removing a license key, the
switch must be rebooted. With no license key, licenseShow displays No licenses.
13
14
15
CMCNE must be installed from the provided CD-ROM or from files downloaded from
Powerlink.
Home > Support > Software Downloads and Licensing > Downloads C > Connectrix
Manager
EMC Connectrix Manager Converged Network Edition
16
The Management application has two parts: the Server and the Client. The Server is installed
on one machine and stores SAN-related information; it does not have a user interface. To
view SAN information through a user interface, you must log in to the Server through a
Client. The Server and Clients may reside on the same machine, or on separate machines.
When installing the Professional version, the server and the client must be on the same
machine.
For more information about system requirements please refer to EMC Connectrix Manager
Converged Network Edition Release Notes.
17
When the installer has been downloaded from Powerlink or the CD has placed into the DVDROM, the installation is ready to run. The installer will guide you through the installation
wizard starting with the Introduction dialog window. Click Next on the Introduction screen
and follow these steps:
18
Once the installation has copied all necessary files, the configuration process can begin. Follow these
steps to configure CMCNE
Check the Welcome screen and click Next.
If you are migrating from a previous version of CMDCE, you can import the previous setting.
If you are installing a new CMCNE instance, Select No and click Next.
Select an option from the Package screen.
SMI Agent is not supported in a Professional edition configuration
Select the CMCNE version to install.
The Enterprise Edition and Professional Plus versions are licensed. The Professional
Edition is included for free with every switch. This version has limited features.
You are not required to enter a license key in this step. If a license is not entered, the
application can be used, including all of its features, for a trial period of 75 days.
However, a license key is required to run the application beyond the trial period.
Consider the following requirements when installing the trial version:
Professional, Professional Plus trial edition and Enterprise trial cannot reside on the
same host unless there are two guest OSs on the same host.
Data collected during Enterprise trial or Professional Plus trial cannot be migrated
back to Professional edition.
After the Enterprise trial or Professional Plus trial, you will need to either roll back to
Professional edition or purchase Enterprise or Professional edition.
When using the Enterprise trial, you will not be able to use a Professional Plus license
in the future.
19
Enter the license key obtained from Powerlink. If you chose the Trial option, select the SAN
with SMI Agent trial of preference. Refer to the EMC Connectrix Manager Converged
Network Edition User guide for more details about Trial version.
Select Internal FTP Server or External FTP Server
Configure the server communication IP addresses
If DNS is not configured for your network, do not select the host name option from
either list. Selecting the host name option prevents clients and devices from
communicating with the Server. If you select a specific IP address from the Server IP
Configuration screen and the selected IP address changes, you are not be to connect
to the server.
Configure the Web Server Port, Database Port and other CMCNE necessary ports on
the Server Configuration screen.
Complete the SMI Agent Configuration.
Select the SAN Network size:
Small (managing up to 2000 ports, 120 domains)
Medium (managing up to 5000 ports, 2160 domains)
Large (managing up to 9000 ports, 61120 domains)
Select the Start Client checkbox on the Start Server screen and make sure that the
Administrative Tools - Services window is closed. If this window is open, the server
may fail to start.
When click Finish, the selected services are started and the Log In dialog box displays.
20
To view SAN information through a GUI, log in to the Server through a Client interface. The
Server and Clients may reside on the same host, or on separate machines.
To log in to a server:
Open the application from the Start menu or double-click the desktop icon.
Enter the user name and password.
Select or clear the Save password checkbox to choose whether you want the
application to remember your password the next time you log in and click Login.
21
The Dashboard tab provides a high-level overview of the network and the current states of managed
devices allowing you to easily check the status of the devices on the network, access reports, device
configuration, and system logs.
The dashboard updates every 5 seconds regardless of the currently selected tab or the SAN size.
However, data may become momentarily out of sync between the dashboard and other areas of the
application.
The Dashboard contains four widgets which can be shown or hidden, resized, collapsed or expanded,
as well as maximized or minimized; however you cannot detach a widget:
1. SAN Operational Status: Displays the device status as a pie chart. Displays the device status as a
percentage of the total number of devices. Displays the percentage in various colors on each
slice. Displays the color legend below the pie chart. Displays tooltips on mouse-over to show the
number of devices in that state. When there is one status category with less than one percent of
the total number of devices, the status widget displays the number of devices in each category on
each slice.
2. SAN Inventory: Displays the SAN products inventory as stacked bar graphs. Displays each group
as a separate bar on the graph. Displays the current state of all products discovered for a group in
various colors on each bar. Displays the color legend below the y-axis. Displays tooltips on
mouse-over to show the number of devices in that state.
3. Events: Displays the number of events by severity level for a specified time range as a stacked bar
graph. you can customize this widget to display a specific time range. Options include: This Hour,
Last Hour, Last 24 Hours, Last 7 Days, or Last 30 Days.
4. Status: Displays the number products managed and the number of events within the selected
event time range. Displays various IP management processes and their current state.
22
The SAN tab is comprised of various areas. Some panels may be hidden by default. To view
all panels, select All Panels from the View menu, or press F12.
1. View All: Enables you to create, copy, or edit a view, select to how to view the Product
list (All Levels, Products and Ports, Products Only, or Ports Only) and to select which view
you want to display in the main window.
2. Port Display buttons: Provides buttons that enable quick access to configuring how ports
display. Does not display until a fabric is discovered.
3. Master Log: The Master Log, which displays in the lower left area of the main window,
lists the events that occurred on the SAN. If the Master Log is not visible, select All Panels
from the View menu.
4. Minimap: The Minimap, which displays in the lower right-hand corner of the main
window, is useful for getting a miniature view of the SAN. To jump to a specific location
on the Connectivity Map, click that area on the Minimap. A close-up view of the selected
location displays on the Physical Map.
5. Connectivity Map: The Connectivity Map, which displays in the upper right area of the
main widow, is a grouped map that shows physical and logical connectivity of SAN
components, including discovered and monitored devices and connections. These
components display as icons in the Connectivity Map.
23
Discovery is the process by which the Management application contacts the devices in a
SAN. When you log in to a Server, the local network is automatically discovered and
displayed on the Connectivity Map. When you configure discovery, the application discovers
products connected to the SAN. The application illustrates each product and its connections
on the Connectivity Map (topology).
The Management application cannot discover a fabric that is in the process of actively
configuring to form a fabric. Wait until the fabric is formed and stable, then re-attempt the
fabric discovery.
After fabric discovery successfully completes, the Management application client that
initiated the discovery is updated to display the newly discovered fabric.
To discover a new Fabric in CMCNE:
24
The Web Tools application can be launched from any workstation with a compatible web
browser installed.
Launch the web browser and type the IP address of the licensed switch in the Address field.
It is necessary to log in before can view or modify any switch information. When the login
screen is presented, you must provide a user name and a password. If Virtual Fabrics are
present, by clicking Options you can choose to log into the home logical fabric or specify a
user logical fabric.
25
The system provides seven default user and admin accounts. Up to 15 additional accounts
can be created per logical switch. Their roles can be designated as either admin or user.
26
The Switch Explorer is the first access you will see when you log in to the switch using Web
Tools. Switch Explorer is divided into areas that provide access to, and information about, the
switch and fabric. The following are the Switch Explorer areas:
1. Tasks: Lets you perform management, monitoring, and other tasks.
2. Fabric Tree: Displays a list of all the switches in the fabric.
3. Menu bar: Provides access to commands and actions. The menu bar displays the same
commands as the left pane of Switch Explorer. If you choose to collapse the left pane,
you still have access to:
27
To view and change general parameters such as name, IP Address and other ones can be
done through the Switch Administration window. To open the Switch Administration
window:
The tabs in advance mode are: Configure, Routing , Extended Fabrics, AAA Service, Trace,
FICON CUP, Security Policies and SNMP.
Most changes made in the Switch Administration window are buffered, and are not applied
to the switch until the changes are saved. If the Switch Administration window is closed
without saving the changes, they are lost. To save the buffered changes, click Apply before
closing the module or before switching to another tab.
The License tab and the Security Policies tab are exceptions. Any changes made on these
tabs are applied immediately and there is no Apply button.
28
It is possible to view and manage ports through the Port Administration window. To access
the Port Administration window, click an accessible port in the Switch View or
29
30
This lesson covers the basic configuration activities for B-Series switches using B-Series and
EMC management tools.
31
A Connectrix B-series must be configured correctly before it can operate within a network
and fabric. If the switch is a director, all of the configuration commands must be entered
through the active CP blade.
32
Once the switch has completed the POST , you must establish a serial connection. Connect
the B-Series serial cable to the Comm port for the switch. Open a terminal emulator
application (such as HyperTerminal on a PC, or TERM, TIP, or Kermit in a UNIX environment),
and configure it to establish the session. When the terminal emulator application stops
reporting information, press Enter to display the login prompt.
Log in using an administrator level account. This allows configuration of the management IP
address.
33
The switch automatically prompts to change the default account passwords after logging in
for the first time. If the passwords are not changed, the switch prompts after each
subsequent login until all the default passwords have been changed.
The default accounts on the switch are admin, user, root, and factory. Use the admin account
to log in to the switch for the first time and to perform the basic configuration tasks.
The change default account passwords prompt accepts a maximum of eight characters. Any
characters beyond the eighth character are ignored. Only the default password is subject to
the eight character limit. User-defined passwords can have 8 to 40 characters. They must
begin with an alphabetic character and can include numeric characters, the period (.), and
the underscore ( _ ). They are case-sensitive, and they are not displayed when you enter
them on the command line.
34
The first task might be assign an IP address to the switch. Use the ipaddrset command to set
static addresses for the Ethernet network interface. Use static Ethernet network interface
addresses on the director and enterprise-class platforms, and in environments where DHCP
service is not available. You may enter static Ethernet information and disable DHCP at the
same time.
35
When executing the initial setup, it is possible to customize the switch name. Switch names
can be from 1 to 31 characters long, except for the ED-DCX-B and ED-DCX-4S-B. These
platforms must be 1 to 15 characters in length. All names must begin with a letter, and can
contain letters, numbers, or the underscore character. It is not necessary to use quotation
marks.
36
Switches maintain the current date and time inside a battery-backed real-time clock (RTC)
circuit. Date and time are used for logging events. Switch operation does not depend on the
date and time; a switch with an incorrect date and time value still functions properly.
However, because the date and time are used for logging, error detection, and
troubleshooting, they should be set correctly.
In a Virtual Fabric there can be a maximum of eight logical switches per director or
enterprise-class platform. Only the default switch in the chassis updates the hardware clock.
When the date command is issued from a non-principal pre-Fabric OS v6.2.0 switch, it will be
dropped by a Fabric OS v6.2.0 (or later) switch and the pre-Fabric OS v6.2.0 switch will not
receive an error.
Authorization access to set or change the date and time for a switch is role-based.
To set the date and time use the mmddHHMMyy syntax.
37
Although Domain IDs are assigned dynamically when a switch is enabled, you can reset them
manually so that you can control the ID number or to resolve a Domain ID conflict when you
merge fabrics. If a switch already has a Domain ID when it is enabled, and that Domain ID
conflicts with a switch already in the fabric, the conflict is automatically resolved. The
process can take several seconds, during which time traffic is delayed.
The default Domain ID for Connectrix B-series switches is 1.
During the same process, it is possible to change other fabric parameters. The following
parameters only can be changed if the switch is disabled, also, they must be identical for a
fabric to merge:
BB Credit
R_A_TOV
E_D_TOV
Switch PID format
Insistent Domain ID Mode
38
Use the configshow command to verify the system configuration settings. The output is
rather long and requires many pages to view in its entirety. You may use the |more switch or
allow the default page length as shown.
39
All licensed ports are enabled by default. On some B-Series models, it is possible to activate
unlicensed ports by purchasing and installing the Ports on Demand optional licensed
product. Before installing a license key, the transceivers must be inserted in the ports to be
activated.
If you enable or disable an active port, the traffic is disrupted and potentially data loss occurs
on that port.
The fabric is reconfigured if the port to be enabled or disabled is connected to another
switch. The switch whose port has been disabled will be segmented from the fabric and all
traffic flowing between it and the fabric will be lost.
To disable or enable a port, connect to the switch and log in using an account assigned to an
admin role.
40
The nsshow command displays local Name Server information, including information about
devices connected to this switch, and cached information about devices connected to other
switches in the fabric. The following message is displayed if there is no information in this
switch:
There is no entry in the Local Name Server.
There still may be devices connected to other switches in the fabric. The command
nsAllShow displays information from all switches. Each line of output shows:
41
The switchshow command displays a switch summary and port summary. Information may
vary by switch model.
The first section provides switch summary information such as:
The second section covers summary information by port. Here you can check the port state
(online/offline), its speed and type.
42
Use the switchstatusshow command to check the overall switch status, along with the status
of each of the following contributors:
Power supplies
Temperatures
Fans
WWN servers (dual CP systems only)
Standby CP (dual CP systems only with HA enabled)
Blades (bladed systems only)
Flash
Marginal ports
Faulty ports
Missing SFPs
43
Under Switch Administration window, use the Network tab to manage the IP networking
functionality of the switch. Remember to ensure a proper IP Address, Subnet Mask, and
Gateway for the switch. When configuring the Ethernet IP, subnet mask, gateway IP, or Fibre
Channel Net IP and subnet mask from Web Tools, there is a normal loss of network
connection to the switch. If the IP properties have changed, close all current windows and
restart Web Tools with the new IP address
The syslog IP represents the IP address of the server that is running the syslog process. The
Syslog daemon reads and forwards system messages to the appropriate log files and/or
users, depending on the system configuration. When one or more IP addresses are
configured, the switch forwards all error log entries to the syslog on the specified server(s).
Up to six servers are supported.
To configure IP and netmask information:
Click Close to exit, and then restart Web Tools to continue working.
44
Switches can be identified by IP address, Domain ID, World Wide Name (WWN), or
customized switch names that are unique and meaningful. Names must begin with an
alphabetic character, but otherwise can consist of alphanumeric, hyphen, and underscore
characters. For most switches, the maximum number of characters is 15.
Many options in a switch are available to change only if the switch is disable. Any option that
is grayed-out is not changeable while the switch is enabled.
45
The Domain ID is a number that uniquely identifies a switch in a fabric. B-Series allows 1-239
for Domain IDs and the default domain ID for switches is 1. Although domain IDs are
assigned dynamically when a switch is enabled, It is possible to request a specific ID to
resolve a domain ID conflict when merging fabrics.
If a switch has a domain ID when it is enabled, and that domain ID conflicts with a switch in
the fabric, the conflict is automatically resolved.
On switches running Fabric OS v4.0.0 and later, do not use domain ID 0. The use of this
domain ID can cause the switch to reboot continuously.
To change the switch domain ID:
46
The Configure tab provides the same functionality as the command line command configure.
The following actions are controllable from the Configure tab:
Fabric Parameters These parameters only can be changed if the switch is disabled
and they must be identical for a fabric to merge as well. It is discussed in the next
slide.
Virtual Channel Allow to configure parameters for eight virtual channels (VC) to
enable fine-tuning for a specific application.
Systems Services Permit enable or disable FCP read link status (RLS) probing for
F_Ports and FL_Ports. It is disabled by default.
Firmware When the firmware is downloaded to a device, the system can validate
the firmware based on a configuration setting. By default, the signed firmware
download is not validated.
47
The following parameters only can be changed if the switch is disabled and they must be
identical for a fabric to merge as well:
BB Credit: Configure the number of buffers available to attached devices for frame
receipt. The default BB Credit is 16. The range is 127.
R_A_TOV: Resource Allocation Time Out value (in milliseconds). This variable works
with the E_D_TOV to determine switch actions when presented with an error
condition. The default is 10000. The possible range is (2*E_D_TOV) -120000. Values
must be multiples of 1000.
E_D_TOV: Error Detect Time Out Value (in milliseconds). This timer is used to flag a
potential error condition when an expected response is not received within the set
time. The valid range is 1000 - (R_A_TOV/2).
Insistent Domain ID Mode: Set this mode to make the current domain ID insistent
across reboots, power cycles, and failovers. This mode is required fabric wide to
transmit FICON data.
48
Use the SNMP tab to specify the switch community string, location, trap level and trap
recipients.
The SNMP tab is affected by the use of Secure Fabric OS. The ACL list is not visible if security
is enabled.
In order for the switches to send SNMP traps, enter the CLI command snmpmibcapset. This
enables the MIBs on all switches to be monitored.
49
The Switch Information Report contains information usually accessed by different CLI
commands but in one single view. The information displayed in this report includes:
50
Under Port Administration window, you can configure FC Ports and GigE Ports.
To configure FC Ports, select the port you want to configure and click Edit Configuration to
open the FC Port Configuration wizard. With the FC Port Configuration wizard, you can
configure allowed port types, port speed, and long distance mode for physical ports.
In Basic mode you can also change:
Port Name - Port names are optional. Ports can be assigned names to make port
grouping easier. The Port Name column in the Ports tab displays the port name, if one
exists.
Enabling or Disabling a Port - All licensed ports are enabled by default. The ports can
disable and re-enable as necessary. To be sure the ports remain disabled through a
reboot the Persistent Disabled button must be checked.
51
The Name Server Table provides the name server entries listed in the Simple Name Server
database. This includes all name server entries for the fabric; not only those that are local to
the local domain. Each row in the table represents a different device.
Click the Name Server in the Monitor section of the tasks menu. The Name Server window
appears.
Brocade has three ways to determine Port ID (FCID). They are:
CorePID 1 This is the preferred method and allows for 255 Ports (00-FF).
0xDDXPAADD = Domain ID, X = Logical Slot Number, P = Port Number, AA = ALPA
CorePID 2 This is a patch to allow newer switches to work with older switches.
0xDDXPAADD = Domain ID, X = Logical Slot Number (Slot 0 and 1 are reversed), P =
Port Number, AA = ALPA
52
Within device properties, it is possible to edit property fields to view/change some switch
properties such as switch name, WWN, IP address between others. Also you can customize
the device Properties dialog boxes to display only the data you need by adding, editing, and
deleting property labels.
Only fields containing a green triangle in the lower right corner are editable. In the slide
above, the only field that can be changed is the switch name.
To edit a field using Properties dialog box:
Right-click the switch and select Properties. The Properties dialog box displays.
Select the tab on which you want to edit a field.
Click in an editable field and change the information.
Click OK.
53
With CMCNE you can enable and disable ports, as well as view port details, properties, type,
status, and connectivity.
To disable or enable a port, right-click the port you want to enable/disable from the Port
Connectivity View dialog box and select Disable/Enable Port.
54
The application displays the percentage of utilization on the trunks as well as on the
utilization legend.
A switchs performance can be monitored through a performance graph, which displays
transmit, receive, and error data from the switch ports to the connected devices. The graphs
show persisted data. Also, the performance of individual switch ports in the SAN may be
monitored through a port performance graph.
Through the application, both high and low usage performance warnings and critical
thresholds can be configured.
Finally, the application allows the user to collect performance data about the SAN and then
export it and distribute the data to others.
To access the monitoring performance tool right-click a switch icon and select Performance
Graphs.
Monitoring performance requires the Performance Monitoring Feature Key to be purchased
and installed into the Connectrix Manager server application.
55
The application records SAN events in the Master Log. The application can be configured to
send event notifications to e-mail addresses at certain time intervals. This is a convenient
way to keep track of events that occur on the SAN.
To configure the application to send notification of events to users, select Monitor> Event
Notification > Email.
56
This lesson covered the basic configuration activities for B-Series switches using B-Series and
EMC management tools.
57
This lesson covers B-Series maintenance tasks such as firmware upgrades and backup and
restore switch configuration.
58
59
Keep a backup copy of the configuration file in case the configuration is lost or unintentional
changes are made. Individual backup files for all switches in the fabric should be kept. Avoid
copying configurations from one switch to another.
To back up a configuration file:
60
Before backup a switch configuration using CLI, verify that the FTP service is running on the
host computer.
User name Enter the user name of the account on the server; for example, root.
File name Specify a file name for the backup file; for example, config.txt. Absolute
path names can be specified using a forward slash (/).
Password Enter the account password for the server.
Store a soft copy of the switch configuration information in a safe place for future reference.
The dnsconfig information and passwords are not saved in a backup.
61
Monitor the progress by looking at the Upload/Download Progress bar on the Configure tab.
62
Before restore a switch configuration using CLI, verify that the FTP service is running on the
host computer.
63
When a firmware download is requested, the system first checks the file size that is to be
downloaded. If the compact flash does not have enough space, Web Tools displays a
message and the download does not occur.
To download a new version of the firmware:
Click Apply.
When the firmware download begins, the firmware download status on the Firmware
Download progress bar shows the download progress.
64
The reboot process takes effect immediately. Ensure that there is no traffic or other
management on the switch, as traffic is interrupted during the reboot; however, frames are
not dropped. Be sure to save your changes before the reboot, as any changes that were not
saved are lost.
A fast boot reduces boot time significantly by bypassing the power-on self test (POST).
Click Apply.
Use the following procedure to reboot the CP and execute the normal power-on booting
sequence.
65
The supportSave command save RASLOG, TRACE, supportShow, and other support
information to a FTP server in interactive mode:
Connect to the switch through a Telnet or SSH utility or a serial console connection.
Log in using an account assigned to the admin role.
Type the supportshow command.
When invoked without operands, this command goes into interactive mode.
66
The supportShow command displays switch information for debugging and diagnostic
purposes. Output generated by it may vary by switch configuration and platform.
Connect to the switch through a Telnet or SSH utility or a serial console connection.
Log in using an account assigned to the admin role.
Set the Telnet or SSH utility to capture output from the screen.
Type the supportshow command.
67
The technical support information can be found in CMCNE Server Console. CMCNE Console is
installed as a stand-alone program which can be accessed from the Start menu. The
Technical Support Information tab of the SMC allows you to capture technical support
information, such as server data for all services. This information is saved in a zip file in a
location that you specify.
To gather technical support information open SMC and perform the following steps:
1. Select the Technical Support Information tab.
2. Click Browse to select the path where the supportShow data will be saved.
If you do not specify an output path, the Management application automatically saves
the data to the C:\Program Files\CMCNE 11.1.1\support directory.
3. Click Capture.
4. A confirmation message displays when the capture is complete.
5. Click OK.
68
The Management application helps you to protect your data by backing it up automatically.
The data can then be restored, as necessary.
The data in the following directories is automatically backed up to disk. The data includes the
following items:
Select an interval from the Backup Interval drop-down list to set how often backup
occurs.
Browse to the hard drive and directory to which you want to back up your data.
Click Apply or OK.
Copyright 2013 EMC Corporation. All rights reserved
69
Open the Server Management Console from the Start menu on the Management
application server.
Click the Services tab. The tab lists the Management application services.
Click Stop to stop all of the services. A CMCNE message appears.
Click Yes. Wait for all four services to stop.
Click the Restore tab.
Browse to the backup location. In order for the restore operation to function properly
the backup directory must be entitled Backup and you must browse one level above
the Backup location. For example, if the backup location is C:\CtxBckup\Backup, then
browse to C:\CtxBckup.
Click Restore. Upon completion, a window displays the status of the restore
operation.
Click Close.
Open the CMCNE application.
70
This lesson covered B-Series maintenance tasks such as firmware upgrades and backup and
restore switch configuration.
71
72
73
SAN configurations using block data include both Fibre Channel (FC) and Internet SCSI (iSCSI)
protocols. Basic security concepts such as authentication, authorization, administration, and
encryption (each explained briefly in this section) are similar for these protocols but mechanisms to
protect Fibre Channel and iSCSI SANs may differ.
Availability is the process of making data accessible in a secured manner. Availability allows data that
resides in a SAN to be available only to authorized end-users, applications, servers, or network
devices when requested.
Authentication is the process of validating the identity of an entity. The authentication process
normally involves a supplicant's presentation of a known credential together with an identifying
element that is either known, possessed, or part of. The strength of the authentication depends on
the number of factors challenged from the above-mentioned list. Authentication in a SAN is
challenged on multiple fronts including switch-switch, host-switch, target-switch, and switch-storage
administration.
Authorization is the process of granting access rights and privileges to an entity that is considered
trusted, usually after authentication is successful. Authorization methods in iSCSI/Fibre Channel SANs
apply to hardware which is the WWN and does not allow changeable usernames. Furthermore, no
secondary checks are made as this would be a weakness that could be exploited through spoofing.
Auditing is the process of capturing and retaining events for current and future analysis. This ability
to capture and retain all events about the infrastructure is essential for security awareness and
overall stability. SAN uses SNMP to trap events and Storage Management Initiative Specification
(SMI-S) to track and manage storage.
Integrity is the process of ensuring that an entity can be trusted whereby it is of the exact form that
was intended. For a SAN, integrity means the preservation of data that is not corrupted by intentional
or unintentional means.
Encryption is the ability to obfuscate an entity, usually data. Encryption is used as a tool to hide
information from unauthorized presentation thereby providing confidentiality. In a SAN, encryption
can be used in two scenarios. Encryption while transmitted across the wire (in-transit); Encryption
within the storage disks (at-rest).
74
Security attacks against SANs are similar to security attacks against IP networks. Breaches of
security can include breaches of authorization, authentication, data confidentiality, and/or
data integrity. iSCSI SANs and Fibre Channel SANs have similar security flaws, including
significant weaknesses with authentication and authorization.
Snooping is a deliberate act to access data without authorization. Different methods include,
but are not limited to, eavesdropping, sniffing, session hacking, intercepting, copying, and
monitoring.
Spoofing is a deliberate act to assume an identity in order to gain unauthorized access to the
data of the company or another user. WWNs are used to identify nodes in a Fibre Channel
SAN, whereas in an iSCSI SAN a node is identified by an iSCSI Qualified Name (IQN). Without
proper security mechanisms in place, both are easy to change and spoof. Some methods
modify part of the information on the fly or use native host bus adapter (HBA) utilities to
change the node identity.
A denial-of-service (DoS) attack is a deliberate act to prevent an authorized user from
accessing data. Limitations of FC and iSCSI SAN protocols can be exploited in order to bring
down the network. For instance, the network interface can be flooded with undesired traffic
or conflicts can be created that cannot be resolved by the SAN, thereby preventing access to
data.
75
B-Series switches provide flexible features to assist you in safeguarding the SAN.
Switch-to-switch (E_Port) authentication using Fibre Channel Certificate Authentication
Protocol (FCAP). Brocade provides commands to disable switch-to-switch FCAP
authentication and to select an alternate authentication protocol such as DH-CHAP.
DH-CHAP is a secrete-based authentication and key management protocol that supports
both switch-to-switch and host-to-switch authentication.
76
Hardware-Enforced PWWN Zoning - The design of the Fibre Channel-switched fabric environment allows
users to dynamically add and remove nodes. When users add or remove a node, hosts are notified of the
change to the fabric environment. Most hosts query the fabric name server to receive an update. Nodes
that do not query the name server may not be aware that their target is no longer available and will
therefore continue to send frames to the same destination port. With hardware-enforced WWN zoning,
the active zoning configuration is pushed to the port ASIC where the ingress and egress ports are located.
Only zone members in the same zone can communicate when they are logged in to the switch. There is no
license required to enable this feature. It is enabled by default and requires no customer configuration.
Persistent Port Disable - Use the persistent port disable command to prevent user-specified ports from
being enabled after a reboot. You can disable any port on the switch through the CLI, Web Tools, or Fabric
Manager. However, when disabled in the normal manner, the port designated as disabled becomes
enabled upon the next reboot. By using the persistent port disable command, the user must manually
enable the port. This feature prevents physically connected nodes from logging in to the switch.
Port Configuration Settings
E_Port Lockout You can prevent any port on a switch from becoming an E_Port by using the
portcfgEport command. By using this setting, other switches and/or fabrics are not permitted to
merge.
L_Port Lockdown If you use the L_Port lockdown command, the switch only initializes the port
as an FL_Port. Fabric loop nodes, such as FC-AL tape drives, would connect to such a port. The
switch does not initialize an F_Port; however, if an attached node commences F_Port
communication, the switch responds and allows the node to log in as an F_Port.
G_Port Lockdown If you use the G_Port lockdown command, the switch does not allow a port
to become either an L_Port or an FL_Port. The port only becomes an F_Port or an E_Port. This
command is set on a port-by-port basis.
77
78
This lab covers SAN discovery, management, and configuration in B-Series switches.
79
This module covered the tools used to manage B-Series switches and the licensing schema.
Basic switch maintenance and configuration tasks are covered, as well.
80
This module focuses on the tools used to manage MDS-Series switches and the licensing
schema. Basic switch maintenance and configuration tasks are covered, as well.
This lesson covers the MDS-Series tools for effective SAN management.
There are multiple connection options and protocols available to manage the MDS Series
switches via the CLI. The initial configuration must be done with VT100 console access.
VT100 console access can be a direct connection or a serial link connection, such as a
modem.
When the initial configuration is complete, you can access the switch with either Secure Shell
(SSH) protocol or Telnet. SSH provides a secure encrypted means of access. Terminal Telnet
access involves a TCP/IP out-of-band (OOB) connection through the 10/100 MB Ethernet port
or an in-band connection via IP over Fibre Channel (FC).
You can access the MDS Series for configuration, status, or management through the console
port, or initiate a Telnet session through the OOB Ethernet management port or through the
in-band IP over FC management feature. The console port is an asynchronous port with a
default configuration of 9600 bps, 8 data bits, no parity, and 1 stop bit.
This port is the only means of accessing the switch after the initial power up until an IP
address is configured for the management port. After an IP address is configured, you can
telnet to the switch through the management 10/100/1000 (previously 10/100) port on the
supervisor card. In-band IP over FC is used to manage remote switches through the local
10/100 port.
The CLI commands are organized hierarchically, with commands that perform similar
functions grouped under the same level. For example, all commands that display information
about the system, configuration, or hardware are grouped under the show command, and all
commands that permit to configure the switch are grouped under the config terminal
command.
To execute a command, start at the top level of the hierarchy. For example, to configure a
Fibre Channel interface, use the config terminal command. Once you are in Configuration
mode, issue the interface command. When you are in the interface submode, you can query
the available commands by typing "?".
To move up one level from Config mode or config sub-mode: type exit.
To move up directly to the top EXEC level: type end.
Fabric Manager (FM) software is downloadable from PowerLink. There are two distinct
versions of Fabric Manager, Standalone and Fabric Manager Server. Fabric Manager Server is
a platform for advanced MDS-Series monitoring, troubleshooting, and configuration
capabilities. This tool provides centralized MDS-Series management services and
performance monitoring.
Fabric Manager Client is a Java and SNMP-based network fabric and device management tool
with a GUI that displays real-time views of your network fabric, including Nexus 5000 Series
switches, MDS-Series switches and third-party switches, hosts, and storage devices.
Fabric Manager Server has the following features:
Roaming user profiles The licensed Fabric Manager Server uses the roaming user
profile feature to store your preferences and topology map layouts on the server, so
that your user interface will be consistent regardless of what computer you use to
manage your storage networks
Cisco is merging two best-in-class management solutions, Cisco Fabric Manager and Cisco Data
Center Network Manager (DCNM) for LAN, into one unified product called Cisco DCNM.
Cisco DCNM can be licensed to manage a combination of SAN and LAN environments. Administrators
still maintain control and segmentation through role-based access control (RBAC), now with singlepane visibility across the network and storage access infrastructure. Depending on licensing,
different features for management of SAN and LAN infrastructure are available.
DCNM assists in operational management of virtualized data centers by providing:
Data center oriented operational dashboards for SAN and LAN health and performance
Proactive monitoring of SAN and LAN health and events/alerts
Performance monitoring and thresholding. Long term trending and capacity planning
DCNM features an innovative new technology called VMPath that provides unprecedented visibility
into the network path starting from a virtual machine through the compute and network (SAN) layers
all the way to storage ports and network services. This, coupled with end to end performance views
enables quick troubleshooting performance bottlenecks and network issues.
Cisco DCNM streamlines the provisioning of the unified fabric by means of template based
provisioning for all of key NX-OS features including that of multi-hop FCoE. DCNM provides
comprehensive coverage for provisioning and monitoring of data center networks built using Cisco
Nexus, Cisco Unified Computing System, and Cisco MDS 9000 Family products, Cisco DCNM is the
Cisco recommended solution for managing mission-critical data centers.
DCNM provides coverage for deploying a wide variety of NX-OS innovations using easy to use wizards
or using templates (for scale, consistency and programmability).
The DCNM Web client allows operators to monitor MDS events, performance and inventory,
and perform minor configuration tasks from a remote location using a web browser.
DCNM Web client provides the following features:
Performance Manager monitors network device statistics historically and provides this
information graphically with a web browser. It presents recent statistics in detail and older
statistics in summary. Performance Manager also is integrated with external tools, such as
Cisco Traffic Analyzer. Performance Manager requires an FMS DCNM Server license.
This lesson covered the MDS-Series tools for effective SAN management.
10
11
A significant change for Cisco DCNM-SAN, as compared to Cisco Fabric Manager, is that
licenses are no longer hosted on a specific switch. Instead, the licenses are hosted on the
Cisco DCNM-SAN server. All existing Cisco Fabric Manager licenses are grandfathered into
this model (which means that they continue) so customers do not need to order or deploy
any additional licenses to manage their existing Connectrix MDS 9000 Family switches.
Two types of Cisco DCNM-SAN licenses are available:
12
The MDS NX-OS (Earlier versions formerly called SAN-OS) runs on all MDS series switches,
from multilayer fabric switches to multilayer directors. Using the same base system software
across the entire product line enables Cisco Systems to provide an extensive, consistent, and
compatible feature set on the MDS-series. NX-OS also runs on the entire Nexus family of
Data Center Ethernet switches, providing a common software infrastructure for the evolution
of unified fabrics.
Most MDS-Series software features are included in the base switch configuration. The
standard software package includes the base set of features that are required by most
customers for building a SAN. However, some features are logically grouped into add-on
packages that must be licensed separately.
13
This slide summarizes the optional features for the MDS-9100, MDS-9200, and MDS-9500
series.
Packages with a blue dot run on the switchs supervisor and are licensed on a per chassis
basis. Packages with text entries run on a service engine, and require one license per service
engine which is running that package. For the MDS 9222i, this class of package can run on
the base MDS 9222i hardware configuration (which has a service engine built in) or on a
module inserted in the MDS 9222is open slot. Note that the base MDS 9222i configuration
comes with SAN Extension capabilities enabled as a standard feature, and does NOT require
a license. If an optional MSM-18/4 or SSN-16 module is added and SAN Extension is required
on that module, licenses are required for that module as usual.
14
Here are some of the most common MDS-9000 series optional packages:
Enterprise package: Adds a set of advanced features which are recommended for all
enterprise SANs.
SAN Extension over IP package: Enables FCIP for IP Storage Services and allows the customer
to use the IP Storage Services to extend SANs over IP networks. Note: FCIP tape read
acceleration is not currently supported by EMC.
Mainframe package: Adds support for the FICON protocol. FICON VSAN support is provided
to help ensure that there is true hardware-based separation of FICON and open systems.
Switch cascading, fabric binding, and intermixing are also included in this package.
Note: FICON tape acceleration and FICON over FCIP are not currently supported by EMC, and
FICON is only qualified on specific versions of SANOS.
FMS (DCNM Server) package: Extends Fabric Manager (DCNM) by providing historical
performance monitoring for network traffic hotspot analysis, centralized management
services, and advanced application integration for greater management efficiency.
Storage Services Enabler package: Enables network-hosted storage applications to run on
the MDS series Storage Services Module (SSM). A Storage Services Enabler package must be
installed on each SSM.
15
License usability can be a nightmare with existing products. Customers have concerns about
compromising availability with disruptive software installations for licensed features. License
management is a notorious problem.
Cisco license packages require a simple installation of an electronic license: no software
installation or upgrade is required. Licenses can also be installed on the switch in the factory.
MDS switches store license keys on the chassis SPROM, so license keys are never lost even
during a switch software reinstall.
DCNM includes a centralized license management console that provides a single interface for
managing licenses across all MDS switches in the fabric, reducing management overhead and
preventing problems due to improperly maintained licensing. In the event that an
administrative error does occur with licensing, the switch provides a grace period before the
unlicensed features are disabled, so there is plenty of time to correct the licensing issue.
All licensed features may be evaluated for a period of up to 120 days before a license is
required.
16
If you need to install multiple licenses in any switch in a MDS Series switch, be sure to
provide unique file names for each license key file. To install a license key file in any switch,
follow these steps:
From a console session on the active supervisor, invoke the install license command:
Exit the switch console and open a new terminal session to view all license files installed on
the switch using the show license command.
If the license meets all guidelines when the install license command is issued, all features
and modules continue functioning as configured. This is true for any MDS-Series switch.
17
The show license usage command can be used to check for licenses installed and in use. Ins
denotes if it is actually installed. If the status displays In use and the Ins shows no, then the
Grace period is bieng used. It starts at 120 days and counts down. When it gets to 0 the
feature is disabled and all configuration data is lost.
18
19
20
Fabric Manager and DCMN cannot be downloaded directly from the switch. It must be
installed from the provided CD-ROM or from files downloaded from Powerlink.
Cisco FM Software Distribution:
21
Once the installer has been downloaded, it must be uncompressed before run the
installation. The start.html file will redirect you to the installation screen, which is the same
screen when running the installation from the CD-ROM.
22
Before run the Fabric Manager installation, ActiveX must be installed and JRE 1.5 or 1.6 is
required (Java 1.6 update is not supported). To install Fabric Manager on Windows:
23
When installing Fabric Manager for the first time, you can choose to install either Express or
Custom Installation. The install defaults to FM Express.
Fabric Manager Express installation option uses Fabric Manager Standalone application and
set all the default parameters. This option uses admin as the user name and password as the
user password. The PostgreSQL database is also installed with admin as the user name and
password_1_2_3 as the user password. The user may change the passwords after the
installation is complete. The Fabric Manager Standalone is a single application containing
Fabric Manager Client and a local version of Fabric Manager Server bundled together. Fabric
Manager Standalone allows the user to discover and monitor the immediate fabric.
The custom installation allows the user to select the Fabric Manager Server (Licensed) option
and the database options available: Oracle10g and PostgreSQL.
The Fabric Manager Server uses considerable system resources and CPU time, it is not
recommended for laptop installation.
24
FM server does not create desktop icons. FM client must be loaded in order for icons to be
created.
To verify that the installation was successful and that the server is functioning, check the
Services window. The service that will be running is Cisco Fabric Manager.
25
Cisco Data Center Network Manager (DCNM) is a management system for the Cisco Unified
Fabric. DCNM streamlines the provisioning for the unified fabric and monitors the SAN and
LAN components. It also supports the installation of the DCNM for SAN and DCNM for LAN
components with a single installer. This course covers only the DCNM for SAN.
Before start the DCNM-SAN installation, ensure ActiveX and JRE and JDK are installed. JRE
and JDK 1.6(x) is supported also Java Web Start 1.5 and 1.6.
Once the installer has been downloaded from Powerlink, it is ready to run. The dcnminstaller.exe guides you through the installation wizard starting with the introduction
message in the DCNM-SAN Installer window.
The wizard request you to select the DCNM-SAN option. Select either DCNM-SAN Server
(Licensed) to install the server components for DCNM-SAN Server or DCNM-SAN Standalone
to install the standalone version of DCNM-SAN. DCNM-SAN Standalone is a single application
containing DCNM-SAN Client and a local version of DCNM-SAN Server bundled together.
DCNM-SAN Standalone allows you to discover and monitor the immediate fabric.
You must also select the database option for the DCNM-SAN installation. The default option
is Install PostgreSQL, however, you can use an existing PostgreSQL installation or an existing
Oracle10g/11g installation.
26
Enter a user name and password in the Local User Credentials dialog box.
In the Authentication Settings dialog box, choose an authentication mode (Local, RADIUS or
TACACS) and click Next. When installing the DCNM-SAN Standalone, you see the
Configuration Options dialog box.
Finally, review the Pre-Installation Summary window and click Next to start the installation.
Once the installation is completed, you see an Installation Completed message in the
DCNM-SAN Installer window.
27
On a Windows machine the DCNM-SAN Server is installed as a service. This service can then
be administered using Services in the Microsoft Windows Control Panel. The default setting
for the DCNM-SAN Server service is that the server is automatically started when the
machine is rebooted. You can change this behavior by modifying the properties in Services.
28
Once DCNM-SAN Server is installed and running, the Web Client can be used. The Web Client
allows interaction thru the server to manage the switches.
To access DCNM Web Client open a connection with a web browser to the IP address of the
DCNM-SAN Server. Log into the Server with the user and password created during the
installation.
29
Add a switch, if there are none, or click the Discover button to add a switch.
Select the Open tab and select a fabric to manage.
All manageable fabrics are listed regardless of their current status.
Managed continuously
Unmanaged
Select the checkbox to view/open fabrics in DCNM.
30
The DCNMs integrated topology mapping capability allows users to visualize zoning
configurations, or examine other relationships between the fabric devices. The Fabric View
window is divided into three major areas:
1. Logical Switch Tree - Organize VSAN, and zoning into folders. Selecting an object from
the switch tree folders displays information about the object in the information pane.
2. Physical Switch Trees - Organize switch information into folders. Selecting an object
from the switch tree folders displays information about the object in the information
pane.
3. Information Pane - Displays tables of switch parameters for objects selected in the
switch tree or Topology Map. The information pane allows users to configure settings
concurrently for multiple switches.
4. Topology Map window - Displays the physical network (fabric), including all end
nodes (N_ports) switches/directors and links. This window also has tabs for viewing
the message log file and a list of SNMP trap events that have occurred since the
Fabric View was opened.
31
Hosts and storage devices, otherwise referred to as initiators and targets, can have
enclosures created for them on the topology map by entering a common name among the
discovered ports. This screen is from the DCNM Physical pane, under End Devices. It displays
information about links to storage in the currently discovered fabric, in a tabular form in the
Information panel. Storage devices show logical unit numbers (LUNs) reported through the
LUN0 inquiry.
The most important information the screen provides is the relationship between the port
World Wide Name (pWWN), the FC identifier (FCID), the name of the device, and the
attached-to information for the switch and interface. This information can assist in
troubleshooting, because you get an overall view of the particular switch interface and what
is connected to it.
Fabric pane can be filtered by group.
Hosts and storage devices
32
33
This lesson covers the basic configuration activities for MDS-Series switches using MDSSeries and EMC management tools.
34
The console needs a rollover RJ-45 cable. There is a switch on the supervisor module of the
MDS 9500 series directors that, if placed in the out position, allows the use of a straight
through cable. The switch is shipped in the in position and is located behind the LEDs.
35
The Basic System Configuration Dialog setup guides you through the basic configuration of
the system. The setup configures only enough connectivity for management of the system,
and it is mainly used for configuring the system when no configuration is present. It is
important to note that the setup command always assumes system defaults and not the
current system configuration values.
The Basic System Configuration Dialog box options can be skipped by entering a carriage
return, or you can skip all remaining dialog boxes by typing Ctrl-C at any time. You must enter
a [y] when prompted to continue with the Basic System Configuration dialog box. The default
password is admin, but this can be changed after initial setup.
After the initial setup is completed, you can log in and make changes to the parameters that
were set during that initial configuration process. If you wish to make changes to the initial
configuration at a later time, the setup command can be issued in the EXEC mode. Then the
setup utility guides you through the basic configuration process.
In the event that you change the administrator password during the initial setup process and
subsequently forget this new password, you have the option to recover this password. You
need to configure only the SNMPv3 user name and password to get access to the switch
through the DCNM. The community strings can be configured at any time.
36
At this point, the name of your switch is entered along with the IP address and subnet mask
of the Ethernet management port interface. Without this information, management access
to the switch through the Ethernet port would not be possible.
When there are options to select with each dialog, you can either press Return, which
accepts the choice indicated between the square brackets (for example, [n]), or you can
select the alternative. In the example, n, for no, was entered at Enable IP routing?,
Configure static route?, and Configure the default network? because [y] was the current
selection and these items were not desired in the configuration. However, Configure the
default gateway? was desired, so pressing Return enabled the user to enter an IP address on
the next dialog line. No other options in the example dialog script were changed.
A Network Time Protocol (NTP) server provides a precise time source (radio clock or atomic
clock) to synchronize the system clocks of network devices. NTP is transported over User
Datagram Protocol (UDP)/IP. All NTP communications use Coordinated Universal Time (UTC).
An NTP server receives its time from a reference time source, such as a radio clock or atomic
clock, attached to the time. NTP distributes this time across the network. Using NTP is
optional but recommended.
Telnet services are enabled to remotely log on to the switch. The DNS client on the switch
communicates with the DNS server to perform the IP address-to-name mapping. Setting up
the Domain Name Server (DNS) is optional but recommended.
37
The system prints a summary of the configuration for your review. The configuration printed
will be exactly what you entered. Compare it once more with the information you obtained
in the initial setup requirements to verify there are no typing errors. If everything was
entered correctly, there is no need to edit.
The system asks if you would like to edit the configuration that just printed out. Any
configuration changes made to a switch are immediately enforced but are not saved. If no
edits are needed, then you are asked if you want to use this configuration and save it as well.
Since [y] (yes) is the default selection, pressing Return activates this function, and the
configuration becomes part of the running-config and is copied to the startup-config.
This also ensures that the kickstart and system boot images are automatically configured.
Therefore, you do not have to run a copy command after this process. A power loss restarts
the switch using the startup-config, which has everything saved that has been configured to
nondefault values. If you do not save the configuration at this point, none of your changes
are updated the next time the switch is rebooted.
38
Connectrix MDS-Series switches use Universal Coordinated Time (UTC), which is the same as
Greenwich Mean Time (GMT). To change the default time on the switch, issue the clock
command from EXEC mode
The clock timezone command sets the time zone with a specified name, specified hours, and
specified minutes.
Use the show clock command to verify the time zone configuration.
39
There are three main types of ports on MDS-Series switches, though each type has its own
subtypes.
An N_Port (node port) is a port on a node that connects to a fabric, for example a host HBA.
I/O adapters and array controllers contain one or more N_Ports. N_Ports can also directly
connect two nodes in a point-to-point or Direct-Attached topology.
An F_Port (fabric port) is a port on a switch that connects to an N_Port.
An E_Port (expansion port) is a port on a switch that connects to another E_Port, in other
words an ISL of some kind. In addition, MDS switches implement TE_Port mode on switches
that connect to other MDS switches and perform VSAN trunking.
40
Enter config
Enter interface fc1/5
Enter switchport mode fx
Enter switchport speed 2000
Enter no shutdown
Enter end
To verify your configuration, use the show interface command: show interface fc1/5 brief.
41
The switchport mode e command configures the interface for E_Port operation. In order to
configure interface fc1/2 as an E_Port, with a speed of 2 Gbps, and trunking disabled, follow
these steps:
Enter config.
Enter interface fc1/2.
Enter switchport mode e (for connections to non-MDS switches).
Enter switchport trunk mode off (for connections to non-MDS switches).
Enter switchport speed 2000.
Enter no shutdown.
Enter end.
The same procedure used to configure an E port must be followed to configure a TE port. In a
TE port, the trunk mode must be enabled. Here is the same example used for interface fc1/2
as TE port:
Enter interface fc1/2.
Enter switchport mode e.
Enter switchport trunk mode on (default).
Enter switchport speed 2000.
Enter no shutdown.
To verify your configuration, use the show interface command: show interface fc1/2 brief
The trunk mode is ignored for any port not configured as an E_Port.
42
43
The shutdown command administratively disables the interface, and the no shutdown
command administratively enables, or activates, the interface.
Follow these steps using Device Manager:
1. Right-click the port.
2. Choose Enable or Disable from the menu.
3. Click the Refresh icon.
44
The show interface brief command displays interfaces and their status. If specific
ports are not specified, then all ports are shown.
To get brief information on a specific range of ports, use the following form of the command:
show interface fc1/1 - 4 brief
Non-contiguous ports can be viewed by separating them with commas as shown below:
show interface fc1/1, fc2/4, fc7/3 brief
45
Using DCNM, in the Physical Switch tree expand the FC Interfaces folder and select Physical.
The administrative state indicates how a specified attribute is configured. For example, in
this slide, if you look at the Mode Admin column, you will notice that all ports are set to FX
(F_Port). The user can configure them as Auto (Configure themselves) or other specific type
such as E port.
The operational state represents the current status of a specified attribute. Some values may
not be valid when the interface is down.
All of the columns in white are interface attributes that the user can configure:
46
Any port in a port group can be allocated 1, 2,4 or 8 Gbps dedicated bandwidth.
All remaining ports in the port group share any remaining unused bandwidth.
Ports in dedicated bandwidth mode have access to a pool of 2488 extended buffers
and 512 performance buffers.
47
48
This slide displays how to set the speed to 4Gbps and the rate-mode for interface fc1/1 in an
oversubscribed FC port module. The displayed example also shows the output of the show
port-resources command. The module will determine how many ports are in a group, and at
which speed can be set. Displayed is module 1 of the 9222i switch. It has three port groups,
each has 6 ports. Every port is defaulted to the shared setting. When a port is set to shared
it can only be an F_port. When a port is changed to a rate-mode of dedicated, the speed
setting is deducted from the total bandwidth. To change a port to dedicated there must be
at least 1 Gbps left in the shared bandwidth pool.
49
Device aliases are independent of the VSAN configuration. Aliases need only be defined once
and can then be used regardless of their VSAN. The device alias configuration and
distribution is independent of the zone server and the zone server database.
The device alias application uses the Cisco Fabric Services (CFS) infrastructure to enable
efficient database management and distribution. Device aliases use the coordinated
distribution mode and the physical distribution scope. You can import legacy zone alias
configurations without losing data.
The device alias databases are preserved across switch restarts and switchovers, and the
system can be restarted or switched over during CFS distributions or merges.
Device aliases have the following requirements:
A device alias name must begin with a letter and is restricted to 1-64 characters.
Permissible characters include one or more of the following:
a to z and A to Z
0 to 9
- (hyphen) and _ (underscore)
$ and ^
Copyright 2013 EMC Corporation. All rights reserved
50
To configure and distribute device aliases using DCNM, choose the End Devices folder in the
Physical Attributes pane. Enter the device alias names as desired in the Device Alias fields in
the Information pane, and then click Apply Changes.
To make device alias names appear as enclosure names in the topology map, highlight the
rows in the table, and click Alias Enclosure.
Device aliases can be also be used in DCNM. For example, when creating zones for a VSAN
using DCNM, the configured device aliases appear for all the pWWNs that have been
mapped to device aliases within that VSAN.
51
This lesson covered the basic configuration activities for MDS-Series switches using MDSSeries and EMC management tools.
52
This lesson covers MDS-Series maintenance tasks such as firmware upgrades and backup and
restore switch configuration.
53
While running, MDS switches have two configuration files. Running Configuration resides in
volatile memory and is not persistent over reboots. Startup Configuration is stored NVRAM
and is persistent over reboots.
Issue the copy running-config startup-config command from EXEC mode to save the new
configuration into nonvolatile storage. Once this command is issued, the running and the
startup copies of the configuration are identical.
To save the Running Configuration use the following command:
# copy running-config startup-config
To save a configuration file to a remote server such as TFTP, FTP, SCP, or SFTP, follow these
steps:
Log into the switch through the console port or through a Telnet or SSH session.
Save the configuration using the copy running-config scheme://url command, where
scheme is TFTP, FTP, SCP, or SFTP and url is the path to the target file on the remote
server.
54
Problems, such as memory corruption, can occur that make it necessary to recover the
configuration from a backed-up version. When restoring a switch, the configuration file is
copied to Startup and then verified using the show run diff command. This command shows
the differences between startup and running configurations. After the file is verified to be
correct, copy start run completes the restore process.
55
To back up the startup configuration in a remote ftp server using Device Manager:
Select Admin > Copy Configuration. The Copy Configuration dialog is opened.
Select the location: startup configuration.
Select the destination of the file: server file.
Enter the server address.
Select the file name.
Choose the file transfer protocol: ftp.
Enter the user name and password for the remote ftp server.
Click Apply to copy the file.
56
To restore the configuration from a remote ftp server using Device Manager:
Select Admin > Copy Configuration. The Copy Configuration dialog is opened.
Select the location: server file.
Select the destination of the file: runningConfig.
Enter the server address.
Select the file name.
Choose the file transfer protocol: ftp.
Enter the user name and password for the remote ftp server.
Click Apply to copy the file.
Note: Once the copy is finished, verify functionality and copy the runningConfig to
StartupConfig.
57
Each Connectrix MDS-Series switch is shipped with a NX-OS operating system. The NX-OS
consists of two images; the kickstart image and the system image.
The MDS 9500 series has an internal bootflash and an external Compactflash. All Flash
devices reside on the supervisor module. The switch software can reside in either of the two
Flash devices.
58
At power-on, only supervisor modules are powered up, and the line card modules stay
powered down. Supervisors jump to the BIOS and start executing the loader from internal
Flash. The loader verifies the kickstart image and loads it. Optionally, the user can boot the
kickstart image through bootp/tftp using the management port, directly from the BIOS or the
loader.
The loader loads the kickstart image from bootflash, based on boot variables or what the
user types in at the loader prompt. The kickstart image contains the operating system (OS),
associated libraries, binaries, and some basic drivers and utilities necessary to boot the
system image. The loader then jumps to the NX-OS and the OS boot starts. After the OS has
completely booted, some basic drivers are loaded.
When boot utilities are fired up and the console starts booting a system image from
bootflash, if no image is found or the image is corrupted or the wrong image type is found,
kickstart stops at the switch(boot)# prompt. If the corruption causes the console to stop at
this prompt, copy the system image and reboot the switch.
Vshboot provides scp and tftp facilities, where the system image is compressed with an MD5
checksum. Kickstart utilities verify the checksum and uncompress and load the system
image. At this time, a component called Platform Manager is loaded. Platform Manager is
one of the first services to be started.
59
To upgrade the switch to a new image, the user must specify the variables that direct the
switch to the images:
To select the kickstart image, use the KICKSTART variable.
To select the system image, use the SYSTEM variable.
The images and variables are important factors in any install procedure. You must specify the
variable and the image to upgrade your switch.
Unless explicitly stated, the software installation procedures in this section apply to any
switch in the MDS series.
The software image installation procedure is dependent on the following factors:
Software images: The kickstart and system image files reside in directories or folders
that can be accessed from the MDS series switch prompt.
Image version: Each image file has a version.
Flash disks on the switch: The bootflash resides on the supervisor and the
CompactFlash disk is inserted into the slot0 device.
Supervisor-1 and Supervisor-2 have specific image requirements:
Sup-1 image filename contains -sf1ek9
Sup-2 image filename contains -sf2ek9
The software installation process is disruptive on systems with a single supervisor module.
60
Before attempting to migrate to any software image version, follow these guidelines:
Before performing any software upgrade, check the EMC Support Matrix to review
requirements and recommendations based on the current operating environment.
Follow required EMC change control procedures.
Schedule the upgrade when the fabric is stable and steady. Ensure that everyone who
has access to the switch or the network is not configuring the switch or the network
during this upgrade, because all configurations will be disallowed at this time.
Verify that sufficient space is available in the location where the images are copied.
This location includes the active and standby supervisor modules or bootflash that is
internal to the switch. You can use the dir command to ensure that the required free
space is available for the image files to be copied. The internal bootflash offers
approximately 200 MB of user space.
Avoid power interruptions to the hardware during any installation procedures. These
kinds of problems can corrupt the software image.
Connectivity to remote servers to retrieve software images requires configuration of
the IP address for the management Ethernet port on the switch (mgmt0) and ensure
the switch has a route to the remote server. The switch and the remote server must
be on the same sub-network if there is no a router to route traffic between subnets.
Verify connectivity to the remote server with the ping command.
The specified system and kickstart images must be compatible with each other. If the
kickstart image is not specified, the switch uses the current running kickstart image. If
a different system image is specified, ensure that it is compatible with the running
kickstart image. To view the current version of system and kickstart images, use the
show version command.
61
The NX-OS provides the ability to upgrade software without any disruptions. To realize the
benefits of nondisruptive upgrades on the MDS 9500 series, it is highly recommended that
you install dual supervisor modules. You can upgrade any switch in the MDS 9100, 9200 and
9500 series using one of three mechanisms:
An automated, one step upgrade using the install all command
A manual step-by-step upgrade
A quick one-step upgrade using the reload command
Before running the reload command, copy the correct kickstart and system images to the
correct location and change the boot commands in config to use them. The quick upgrade,
however is disruptive. For nondisruptive upgrades, use the automated one-step upgrade or
the manual step-by-step upgrade.
In some cases, any software upgrade may be disruptive. These exception situations can occur
under the following conditions:
A single supervisor system with kickstart image changes
A single supervisor system with incompatible system software images
A dual supervisor system with incompatible system software images
The install all command compares and presents the results of the compatibility before
proceeding with the installation. You have the opportunity to exit if you do not want to
proceed with these changes. To determine version compatibility between switch images, use
the show install all impact command. This command displays the impact of using the install
all command.
62
It is recommended that the one-step install all command be used to upgrade the system
software. This command upgrades all modules in any MDS series switch. Only one install all
command can be running on a switch at any time, and no other command can be issued
while running that command. The install all command can not be performed on the standby
supervisor module. It can only be issued on the active supervisor module.
If the switching modules are not compatible with the new supervisor module image, some
traffic disruption may be noticed in the related modules, depending on your configuration.
These modules are identified in the summary when you issue the install all command. You
can choose to proceed with the upgrade or abort at this point.
To save the configuration after an upgrade, enter the copy running-config startup-config
command from the executive mode prompt to save the configuration into non-volatile
storage. After this command is issued, the running and the startup copies of the
configuration are identical. This is necessary because the boot variables are not
automatically instantiated to the startup config file after an upgrade (manual, install all, GUI,
and so on). Without issuing this command immediately after a successful upgrade, you run
the risk of the switch rebooting to a previous release.
It is recommended that you issue the install all command from the console terminal of the
active supervisor module while having an additional console terminal open for the standby
supervisor module. After the install all command is issued in the console terminal of the
active supervisor module, the console terminal of the active supervisor module does not
close, but it does become the console for the new standby supervisor. The standby
supervisor module console terminal remains open and displays the full set of messages
printed by the install all command process. If you choose to issue the install all command
from a telnet or secure shell (SSH) session, you can not view the full set of messages because
the connection is lost when the supervisor switchover occurs.
63
Local, where images are locally available on the switch. The install all command uses
the specified local images.
Remote, where images are in a remote location and the user specifies the destination
using the remote server parameters and the file name to be used locally.
To upgrade the switch to a new image, specify the variables that direct the switch to the
images. To select the kickstart image, use the kickstart variable, or to select the system
image, use the system variable. The images and variables are important factors in any install
procedure. You must specify the variable and the image to upgrade your switch. Both images
are not always required for each installation.
64
When you issue the install all command, the switch displays a summary of changes that are
made to your configuration and waits for your authorization to continue executing the
command process.
A compatibility check is conducted for each module installed in the system to be upgraded.
The impact of an upgrade and the install type are displayed.
Modules and specific images to be upgraded based on the files specified in the previous step
are displayed in an upgrade table that also shows the running and new versions.
Compatibility check terms are as follows:
Bootable: The ability of the module to boot or not boot, based on image
compatibility
65
If there are no issues with compatibility and you wish to continue with installing the images
identified, enter y (yes) when prompted.
After you confirm to continue with the installation, a display of the installation progress
appears. If install succeeds, you receive a notification that the install has been successful and
the switch prompt is displayed.
Some benefits of using the install all command to upgrade system software are:
The streamlined process ensures using the best possible procedures to upgrade the
software in the least disruptive manner.
66
To upgrade the switch firmware of one or more MDS 9000 switches using DCNM:
Note: If the files are to be downloaded during the install, also enter the path and filename of
the images.
Check the Skip Image Download check box. This enables an upgrade to be performed using
images that are already located on the supervisor's bootflash. Depending on the installation
method, the wizard may prompt for additional file locations. The fourth and final screen
provides a summary and enables the installation to begin.
67
To downgrade from a higher release use the install all command to gracefully reload the
switch and handle configuration conversions. When downgrading any switch in the MDS
series, avoid using the reload command.
Downgrading from NX-OS Release 4.2(7b) or later to SAN-OS Release 3.3(1c) and earlier is
disruptive and requires a reload of the switch.
See Determining Software Compatibility in the Cisco MDS Series Configuration Guide for
more details.
68
This lesson covered MDS-Series maintenance tasks such as firmware upgrades and backup
and restore switch configuration.
69
70
Many factors need to be considered when managing SANs. Network and security
requirements are often unique to each business environment. As The FC SAN tends to be an
isolated network, The host OS, IP Connectivity and the management interfaces are the most
likely points of intrusion. These aspects are subject to well known hacks and exploits.
Security attacks against SANs are similar to security attacks against IP networks. Breaches of
security can include breaches of authorization, authentication, data confidentiality, and/or
data integrity. iSCSI SANs and Fibre Channel SANs have similar security flaws, including
significant weaknesses with authentication and authorization.
71
MDS switches and directors provide a comprehensive security framework within NX-OS and
SAN-OS. Licensing is required for some enhanced security features including FC-SP
authentication, port security, LUN zoning, IPSec, and VSAN-based access control.
For a listing of EMC conditionally or unsupported product features, please refer to the EMC
Support Matrix:
72
SAN management consoles are primary targets for attackers. Risks include usage of clear text
management protocols, weak username and passwords, un-segmented communication networks,
and shared accounts. Administrators should deploy strong authentication and authorization
mechanisms to secure SAN management. Implementation decisions are necessary to secure SAN
management functions while balancing business needs for accessibility and performance.
Unauthorized or unintentional access to SAN management can jeopardize the integrity and stability
of the SAN infrastructure. Traditional access protocols such as Telnet, rlogin, SNMPv1, SNMPv2 and
FTP are inherently insecure when used to access management ports on the MDS.
SSHv2 helps to prevent man-in-the-middle or replay attacks by providing an encrypted access link
between the management client and the switch. SSHv2 encrypts traffic between client and MDS
Series, authenticates communication between client and host, and prevents unauthorized access.
However, you must configure SSH Host Key Pair before enabling the SSH service.
There are three key pairs:
Rivest, Shamir, and Adelman (RSA1) for Secure Shell version 1 (SSHv1) protocol
Digital Signature Authority (DSA) for SSHv2 protocol
RSA for SSHv2 protocol
With SSH, no password prompt is given. SSH is useful when running scripts. You need to first generate
the SSH key-pair on the SSH client machine, then configure the public key on the MDS switch. You
need to create a user account before you can configure an SSH key.
73
Switches in the MDS Series perform authentication based on roles. Role-based authorization limits
access to switch operations by assigning users to roles. This kind of authentication restricts users to
management operations based on the roles to which they have been assigned. When you execute a
command, perform command completion, or obtain context sensitive help; the switch software
allows the operation to progress only if you have permission to access that command.
By default, two roles exist in all MDS switches:
Network-operator: This person has permission to view the configuration only and cannot
make any configuration changes.
Network-admin: This person has permission to execute all commands and make
configuration changes.
The administrator can also create and customize up to 64 additional roles. Up to 16 rules can be
configured for each role. Only users belonging to the network-admin role can perform commands
related to roles.
If you use a SAN Volume Controller (SVC) setup, two more default roles exist in all MDS switches:
Svc-admin: This person has permission to view the entire configuration and make SVCspecific configuration changes.
74
DCNM can be used to create simple roles across multiple switches in the fabric. To create a
role using DCNM, first choose SNMP from the Security folder in the Physical Attributes pane,
and then click Create Row in the toolbar. A creation dialog box appears. Click the checkboxes
for the switches where you want to configure the role, and enter a name and description for
the role.
The granularity of the roles created using DCNM is limited. You can check or uncheck the Has
Config and Exec Permission checkbox. If you uncheck the box, the role basically has readonly permissions. You can check the VSAN Scope Enable checkbox to restrict role to one or
more VSANs.
75
IP Access Control Lists (IP ACLs) provide basic network security to all switches in the MDS
Series. IP ACLs restrict IP-related MDS out-of-band management traffic and in-band traffic
based on IP addresses (Layer 3 and Layer 4 information). You can use IP ACLs to control
transmissions on an interface.
Follow these guidelines when configuring IP ACLs in any switch or director in the MDS Series:
76
The fabric binding feature ensures ISLs are enabled only between specified switches in the
fabric-binding configuration. Fabric binding is configured on a per-VSAN basis. This feature
helps prevent unauthorized switches from joining the fabric or disrupting current fabric
operations. Fabric binding requires that you install either the MAINFRAME_PKG license or
the ENTERPRISE_PKG license on your switch.
77
Use port controls to eliminate the dangers of having users, intentionally or not, misuse a port
that has the default 'auto' mode port settings. To avoid this danger, configure 'port' mode on
all switch ports, shut down all used ports, and only allow connections from expected device
types by specifying N_Port, E_Port, F_Port, and FL_Port settings.
Port security prevents unauthorized access to a switch port by binding specific WWN access
to one ore more given switch ports. Complimentary to port security is WWN-based zoning
which zones the switching logic to frames based on the WWN, and not the physical port, on
a device. With this logical security, spoofing can be a problem.
When enabling Port Binding, consider the impact of choosing whether or not device-toswitch or switch-to-switch port security is enabled and assure that it does not impact
something that should be accessing a specific port. When these features are enabled, it
rejects login requests from unauthorized FC devices, as well as, reports attempts to the SAN
administrator.
The port security feature uses the Cisco Fabric Services (CFS) infrastructure to enable
efficient database management, provide a single point of configuration for the entire fabric
in the VSAN, and enforce the port security policies on throughout the fabric. To enforce port
security, configure the devices and switch port interfaces through which each device or
switch is connected, and activate the configuration.
By default, the port security feature is not activated in any switch in the Cisco MDS 9000
Family.
78
79
This lab covers SAN discovery, management, and configuration in MDS-Series switches.
80
This module covered the tools used to manage MDS-Series switches and the licensing
schema. Basic switch maintenance and configuration tasks were covered as well.
81
82
This module focuses on managing all aspects of a SAN from hosts to storage to the actual
connectivity.
This lesson covers the N_Port initialization process, as well as fabric login.
There are three types of logins supported in Fibre Channel; Fabric, Port and Process. All
node ports must attempt to log in with the Fabric. This is typically done right after the link or
the Loop has been initialized.
When a device is physically connected to a fabric switch port, the Fibre Channel protocol
establishes a logical connection between the node and the fabric switch. This is called Link
Initialization. Primitive Ordered Sets are sent between the node and the switch to establish
the link.
Once the physical link is established, the node sends a FLOGI frame to the port to allow it to
communicate with the rest of the fabric through the Fabric Login. This frame is received by
the login server which responds back with an assigned FCID.
Next, each node performs a Name Server registration. The Name Server obtains information
from the node through the port login frame and through subsequent registration frames.
Information in the Name Server is stored in the form of database objects. The node may
register values for all or some database objects depending on the requirement. The most
commonly registered objects are: fabric address, WWPN, WWNN, and ULPs, Classes of
service supported, and Port type. The Node also requests a list of nodes that support the
same FC-4 Upper Layer Protocols as itself. This list usually depends on whether there are
restrictions placed on which devices the node can talk through zoning.
Fibre Channel IDs (FCIDs) are assigned by a switch when the devices (Nx ports), including
hosts, disks, and tape arrays log in to the fabric. FCIDs can therefore change as devices are
removed from and added to the fabric. On older switches and code levels FCIDs were
dynamically assigned by default but now most modern switches and updated code levels
FCIDs are assigned persistently by default, meaning that the same FCID is always assigned to
a given port World Wide Name (pWWN).
The FLOGI exchange contains the WWN of the N port in the payload of one of its frames. The
switch then sends an ACC reply that contains the N port address (FCID) in the destination ID
field.
Some operating systems, such as Hewlett-Packard UNIX (HP-UX) v11.0 and prior, and IBM AIX
v5.2 and prior, map block devices, such as file systems, by default to the assigned FCIDs. As
each Fibre Channel target device is attached to the operating system, the FCID is used as the
identifier, not the WWN as in many other operating systems.
The problem with the target-binding method employed by legacy HP-UX and AIX is there are
several possible cases in which a new FCID may be assigned to a storage device, thereby
invalidating the binding held by a given server. These cases may involve a simple move of a
storage device, or perhaps a port failure requiring the storage device to be moved to a
different switch port. It could even be something as simple as a SAN switch being rebooted.
All of these conditions could cause new FCIDs to be assigned to existing storage devices.
A SAN designer must pay very close attention to this detail when deploying legacy HP-UX and
AIX-based servers in a SAN as this binding method can represent a significant high availability
risk.
IBM AIX v5.2 and later versions include a new feature called dynamic tracking of FC devices
that can detect the change of a target FCID and remap the target without any intervention.
HP-UX 11iv3 provides a feature called agile addressing which no longer uses the target FCID
as part of the path.
On current switches the persistent FCID feature is enabled by default. This prevents FCIDs
from being changed after a reboot, or when a device moves to another port within the same
switch and VSAN.
When an N port logs in to the switch, it is assigned an FCID. The currently assigned FCIDs are
saved across reboots, and the fcdomain database is automatically updated with dynamic
entries that the switch learns about when new devices log in to the switch. Also when that
device is moved to another port within the same switch and VSAN the same FCID is assigned.
As a general best practice recommendation, switch domain IDs should always be statically
assigned. This is particularly important whenever you wish FCIDs to persist, because a
change in the domain ID means that the corresponding FCIDs must change. This is because
the first eight bits of the FCID must be the domain ID of the switch, VSAN or VF.
The node then attempts a PLOGI to all nodes from the list it receives from the switchs Name
Server. It provides a specific set of operating characteristics associated with the destination
N_Port, Classes of Service are supported. It also initializes the destination end-to-end credit.
The process is repeated as other nodes are attached to other ports on the switch.
Finally, the node then sets up the environment between itself and the device its
communicating with by performing a Process Login. This environment is then used to
determine if there is a LUN present. This is the point at which storage connectivity is
established. A group of related processes is collectively known as an image pair. The
processes involved can be system processes, system images, control unit images or ULP
processes. The use of process login is required by a specific upper-level protocol such as
SCSI-FCP mapping.
This lesson covered the N_Port initialization process as well as fabric login.
10
This lesson introduces WWNs. It focuses on showing students how to find the WWNs of your
hosts and analyzing the nameserver.
11
Each device attached to a fabric switch has a unique 64-bit identifier called a World Wide
Name (WWN). These names are factory-set on HBAs, and are software-generated for storage
ports. In a WWN, the first four bits identify the format being used. A WWN is either a factory
setting on physical devices such as HBAs or FAs. A WWPN may also be a software-generated
identifier.
Values for worldwide name formats are based on the IEEE company ID. More information on
these formats can be found at http://www.standards.ieee.org
A worldwide name has two components:
Worldwide Port name (WWPN)
Worldwide node name (WWNN). The WWNN can be used to identify a unit
containing a group of ports.
The WWNN is neither universally nor consistently used by the industry. A host may have one
WWNN for the entire unit, but several WWPNs (one for each port).
A WWN consists of eight hex pairs separated by colons, for example 10:00:08:00:88:44:50:ef.
The Fibre Channel address of a device may change, but its WWN cannot change. Its location
in a fabric can be easily tracked.
12
The first digit of a WWN is the IEEE code that identifies the type of WWN. A 5 in the first column indicates that
the WWN is an IEEE registered name. Most HBA WWNs begin with a 1, which indicates a standard IEEE 48-bit
ID. The next 24 bits (six characters) is the companys OUI (Organizational Unique Identifier). In this example,
EMCs OUI (006048) is given. The remaining bits are used by manufacturers to uniquely identify hardware.
In a Symmetrix, the 30 bits following EMCs OUI are the Symmetrix Serial Number. In older Symmetrix models,
the last six bits are used to identify the SP and port.
The DMX has more ports. It uses the same calculations along with a side bit calculation:
The WWPN of an FA port is generated using the Symmetrix serial number and the card slot. As a result, the
WWPN stays constant if the FA has to be replaced. The method used to derive the WWPN of an FA depends on
the version of the Symmetrix.
On a CLARiiON, SP port names are dynamically assigned, so they do not change if components are replaced.
WWPN
Bits 63-60 - IEEE WWN type (0x5)
Bits 59-36 - CLARiiON company ID as assigned by IEEE (0x006016)
Bits 35-32 - This value differentiates multiple ports on the same array (node name).
It can range from 0-15. These values are always the same per model
Bits 31-0 - This value is the 32-bit WWN seed read from resume PROM.
13
Emulex HBAs generally start with 10000000c9, followed by the company specific
information.
QLogic HBAs generally start with 200000e88b, followed by the company specific information.
HP HBAs start with 5006B0, followed by the company-specific information.
Also, keep in mind that most vendors have multiple OUI strings to allow for more products.
14
From the AIX host, find the fibre channel adapters installed by running the command:
lscfg vl fcs0
The part number and FRU (Field Replaceable Unit) number are valuable for downloading the
microcode used to upgrade the HBAs. The WWPN of the HBA is displayed under Network
Address as well as the current microcode level.
15
The following steps describe how to determine the HBA WWN on an HP-UX host. To
determine an HBA WWN, you must first display devices attached to that host. Do this by
running the command ioscan -fnC fc or running the sam utility and determining the HBA's
device name. Once you have this you can use the fcmsutil <device name> command to list
details about that HBA. The output displayed includes HBA port and node WWNs.
16
From the Solaris host, find the fibre channel adapters installed by running the fcinfo
command.
Solaris 10 and above use the StoreEdge SAN Foundations drivers (also known as Leadville
driver), natively installed with the OS. This new drivers significantly simplify the HBA
installation and configuration process. To get the HBA model, firmware, bios and WWPN for
all the HBAs simply type this command.
17
The Emulex utilities include hbanyware, which is a GUI for managing HBAs and its CLI
version, hbacmd. When writing a command for a specific HBA the HBA WWN is used to
identify it. The first step when using hbacmd is to look for the HBA WWNs, do this by running
the following command:
/usr/sbin/hbanyware/hbacmd listhbas
Once the PWWN for the HBAs is located, other commands can be executed to provide more
information, such as:
/usr/sbin/hbanyware/hbacmd HBAAttrib 10:00:00:00:c9:4e:a9:e2
18
From the Windows host, find the Emulex fibre channel adapters installed by opening the
OneCommand Manager utility GUI from the Desktop or Start, Programs, Emulex. From the
utility expand the driver model and select the WWN. The Port information tab shows
valuable information such as the WWN, Boot version, Driver version and firmware version.
The utility allows Diagnostics and Statistics as well as setting attributes under Driver
Parameters.
19
It is recommended to use 4 GB or 8 GB HBA cards starting at ESX 3.x and higher to achieve
better performance.
To check QLogic HBA parameters, issue the following command:
# /proc/scsi/qlaxxx/#
For Emulex HBAs:
# /proc/scsi/lpfcxxx/#
Where # is the sequential value of each QLogic HBA installed in the system, beginning with
the number after the last host adapter number entry in the file. The parameters contain
useful information of the initiator, the major information including:
HBA Model Number
Driver, Firmware and Bios Version
Current Speed
Link Down Timeout Value
Port Down Retry Times
WWPN/WWNN of initiator
WWPN/WWNN of the target being connected
20
Web Tools displays Name Server entries listed in the Simple Name Server database. This
includes all Name Server entries for the fabric, not only those related to the local domain.
Each row in the table represents a different device. You can click the column head to sort the
events by a particular column, and drag the column divider to resize a column. You can also
right-click a column heading to resize one or all columns, sort the information in ascending or
descending order, or select which columns are displayed.
Admin Domain considerations: The Name Server table is filtered based on Admin Domain
membership of the fabric devices. The Name Server table lists only devices that are part of
your current Admin Domain. This includes devices that are direct members of the Admin
Domain and devices that are attached to ports that are direct members of the Admin
Domain. All other fabric devices are filtered out of the Name Server view for the current
Admin Domain.
21
Device Manager provides name server information as well. Log into Device Manager and
from the menu select FC > Name Server.
The name server table is displayed with all the devices logged in to the fabric. This windows
provides the ability to save or print the output of the name server.
22
23
Zoning is the process of grouping initiator and targets into zones. Initiators and targets placed
in the same zone are allowed to communicate by the fabric. Zoning also plays a critical role in
event isolation, i.e., RSCN (Registered State Change Notifications) distribution. In order for a
fabric to work properly, it must be properly zoned.
A collection of zones is called a zone set. The zone set can be active or inactive. An active
zone set is the collection of zones currently being used by the switched fabric to manage
data traffic.
Single HBA zoning consists of a single HBA port and one or more storage ports. It is
important to note that a port can reside in multiple zones, which provides the ability to map
a single Storage port to multiple host ports. For example, a Symmetrix FA port or a CLARiiON
SP port can be mapped to multiple single HBA zones. This allows multiple hosts to share a
single storage port (for example, hosts in a cluster). Single HBA zoning simulates a single
initiator SCSI environment. This reduces risks associated with issues such as different driver
revisions, HBA types, and heterogeneous servers on the same fabric. Best practices
recommend a single HBA zoned to a single storage port.
24
By far, the most common form of zoning is by WWPN. You select the unique 64-bit addresses
of the initiator, its target(s), and place them in a common zone. Its advantage is that no
matter where you attach the WWPN, as long as it is in the same fabric, it will always be able
to discover, and be discovered, by other ports to which it has been zoned to have access. Be
aware that an interface has World Wide Node Name. and WWPN. WWPN refers to the port
on the device, while WWNN refers to the overall device. For example, a dual-port HBA has
one WWNN and two WWPNs. Always use WWPN identification instead of WWNN.
The Domain, Port (D,P) is another common form of zoning. D,P uses the switch domain ID
and switch port to identify zone members When zoning, you specify an initiator and target by
their physical location in the fabric. The main advantage is perceived to be security, and in
some cases it is more secure.
While WWPN and D,P can be mixed in a Zoning configuration or a zone, this can lead to
configuration errors, because different processes are required for each method. For this
reason and for an easy operation, it is not recommended to use hybrid zones.
25
This slide describes the tasks involved in setting up a zone. Planning is essential to ensure
ease of troubleshooting and reconfiguration involved when components are replaced or
added. Each vendor has a set of recommendations for setting up zones.
26
This lab covers how to discover host WWNs and match them with the storage name server.
We also decode Storage WWNs with E-Lab Advisor and create a map of the setup
27
This lesson covered WWNs, focusing on showing students how to find the WWNs of hosts
and analyze the nameserver.
28
This lesson covers B-Series Virtual Fabrics, navigating through key terminology and
configuration.
29
Physical switches can be partitioned into independently managed Logical Switches each with
their own data, control, and management paths. Logical Switches can allocate fabric
resources by the port rather than by the switch. They also provide a way to simplify
charge-back for storage by customer, department, or application while cost-effectively
consolidating SAN resources. Because Logical Switches do not need to be enabled on every
switch in a SAN, deployment is simple and non-disruptive in existing environments.
A Logical Fabric is an implementation of a Fibre Channel fabric with one or more Logical
Switches participating in the fabric. A Logical Fabric has its own independent instance of
fabric services, name server, zoning database, and so on. Events in one Logical Fabric are
isolated from events in another.
Virtual Fabrics cannot be configured or managed from Web Tools. Configuration and
management is done from either the CMCNE, or the Fabric OS command line interface.
You can use Web Tools to view Virtual Fabrics and logical switch configurations.
30
There are three types of Logical Switches that can be created on a physical switch; one is
created by the system automatically and the other two are user-defined.
The Default Switch is automatically created when Virtual Fabrics is enabled and cannot be
removed without disabling Virtual Fabrics. There can be only one Default Switch per physical
switch. The Default Switch has a Fabric ID (FID) of 128. When Virtual Fabric is enabled, all
ports on that switch are allocated to the Default Switch. These ports can later be allocated
to any other switch on the same physical switch.
The Base Switch is an optional user-configured switch that is used to provide communication
and connectivity across physical switch chasses and/or between Logical Fabrics. The Base
Switch is not meant to connect devices, and any F or FL ports that form on a Base Switch will
be disabled.
All other Logical Switches are considered standard or general Logical Switches (referred to as
Logical Switches in this module). The properties of these switches are completely
customizable by the administrator.
31
A logical fabric is a fabric that contains at least one logical switch, however, users can form
logical fabrics connecting logical switches to non-Virtual Fabric switch and to other logical
switches. There are two ways to connect logical switches to other logical switches; using
DISLs and, using base switches and sharing ISLs.
A Logical Fabric can connect to a physical switch by inserting an ISL between a Logical Switch
and the physical switch.
32
Each end of the XISL is connected to a port that is a member of the Base Logical Switch. XISLs
can reduce the number of ports that need to be allocated for inter-switch connectivity.
Additional information is placed in the FC Frames that cross XISL links to identify which
Logical Fabric the data is to be delivered to. XISLs can also utilize port trunking, as well as ICL
(Inter-Chassis Links) ports on the ED-DCX-B and ED-DCX-4SB. There are several restrictions
on the use of XISLs, which are listed here. Please note that XISLs do not provide
communication between Logical Fabrics, only between Logical Switches in the same Logical
Fabric.
By default, the physical ISL path is favored over the logical path over the XISL because it has a
lower cost. This behavior can be changed by configuring the cost of the dedicated physical ISL
to match the cost of the Logical ISL (LISL)
33
Virtual Fabrics can be enabled or disabled from the CLI of a switch by using the fosconfig
command with the appropriate parameter. It can also be enabled within CMCNE. Enabling or
disabling VF on a switch impacts only that switch; it does not enable or disable VF on any
other switch in the fabric/Virtual Fabric. If the disable command is issued with any Logical
Switches still defined, except for the Default Switch, an error is generated. All Logical
Switches except the base switch must be deleted prior to disabling VF. The switch will reboot
whenever VF is enabled or disabled.
When Virtual Fabrics are enabled, the switch prompt changes to reflect the FID context the
user is currently in, as shown here. If the setcontext command is used to change from
one Logical Switch to another, the prompt will also change. This provides an easy reference
to know which Logical Switch you are currently managing.
Enabling Virtual Fabrics is done at a switch-wide Level and requires a reboot to take effect.
Running the fosconfig enable vf command is also the first point at which you are
warned if this switch is not compatible.
34
lscfg is the primary CLI command that is used to configure and manage Logical Switches.
This command can be used to create or delete Logical Switches, add or remove ports from a
switch or change the Fabric ID of a Logical Switch. This command requires Chassis
Management permissions to execute.
35
To create a new switch, the lscfg --create command is used. There is one required
parameter, the Fabric ID. This is a number between 1 and 128 similar to the VSAN ID in an
MDS-Series switch. If a value that is already in use is specified, the command will fail.
Switches are automatically enabled and have zero ports initially.
In addition, there are two optional arguments that can be used: -b which configures the new
Logical Switch as the Base Switch; and -f which executes the command without a
confirmation prompt.
36
The lscfg --config command is used to move ports from one Logical Switch to
another. This command has required parameters:
37
Here we see the output from the lscfg --show command. We see that there are
currently two logical switches: FID 128 (the Default Switch) as well as FID 10 which is a logical
switch we created on the pervious slides. Notice the port assignment for logical switch 128
and 10.
38
A base switch is in charge of containing the ISLs between VF-enabled B-Series switches. In
order for a logical switch to be promoted to base, XISL use must be disabled. This can be
done by running the configure command as shown above. Once XISL use is disabled, the
logical switch can be changed to become base, this operation is disruptive to the logical
switch since it is disabled. When the operation is complete enable the logical switch. We will
be using the base switch to connect B-Series switches together in a fabric.
39
This lesson covered B-Series Virtual Fabrics, navigating through key terminology and
configuration.
40
This lesson covers zoning tasks in B-Series CLI, Web Tools and Connectrix Manager CNE.
41
The most commonly used zoning commands are shown in this table.
42
43
The cfgShow command displays the saved configurations and the effective configuration
being enforced. There can be many saved configurations but only one can be active at any
given time.
The zone --validate command is used to verify the effective configuration. Its output
is very similar to the cfgShow command with the added benefit of checking the zone
members to make sure they are logged in the fabric. In the example above we can see an
error condition exists in our VMAX port, the member does not exist in the fabric. Further
investigate this issue by checking the name server database, the port where the storage
array is connected and the physical connectivity.
44
45
The Zoning modes is the first view provide by Zone Administration window. It is possible to
use two modes:
Basic zoning: The basic zoning permit to partition a SAN into logical groups of devices
that can access each other. Zones can be configured dynamically. They can vary in
size, depending on the number of fabric-connected devices, and devices can belong
to more than one zone. Because zone members can access only other members of
the same zone, a device not included in a zone is not available to members of that
zone.
Traffic Isolation zones (TI zone): It is a special zone that creates a dedicated path for a
specific traffic flow. TI zones are primarily for shaping and controlling traffic rather
than partitioning access to storage.
46
Use the Zone tab to create, modify, rename, or delete zones in the zoning database.
To create a zone:
47
1. Click + signs in the Member Selection List to view the nested elements. The choices available
in the Member Selection List depend on the selection made in the View menu (Zoning
Methods).
1. Zone membership can include ports, WWNs, aliases or any combination of these. A
device can be included in more than one zone.
2. To specify by port number, specify the switch domain ID and area number. For example,
2,12 indicates switch domain ID 2, area number 12. When a member is specified by port
number, all devices connected to the port are included in the zone.
3. To specify by WWN, specify node name or port name as an eight byte hex number
separated by colons, for example, 10:00:00:00:60:69:00:8a. These eight numbers are
compared to the node and port name presented in a login frame (FLOGI or PLOGI).
48
A zone configuration is a group of zones that are enforced whenever that zone configuration is
enabled. There are different configuration types:
Defined Configuration The complete set of all zone objects defined in the fabric.
Effective Configuration A single zone configuration that is currently in effect.
Saved Configuration A copy of the defined configuration plus the name of the effective
configuration, which is saved in flash memory by the cfgSave command
Disabled Configuration The effective configuration is removed from flash memory.
Use the following procedure to create a zone configuration. After creating a zone configuration,
enable it for it to take effect.
1. Click the Zone Config tab.
2. Click New Zone Config (The Create New Config dialog box appears).
3. Type a name for the new configuration and click OK. The zone configuration name must be a
unique alphanumeric string beginning with an alphabetic character and cannot exceed 64
characters .
Now, add the zones to the zone configuration
1. Click + signs in the Member Selection List to view the nested elements.
2. Highlight an element in the Member Selection List to be included in the configuration.
3. The Add Member button becomes active.
4. Click Add Member to add configuration members.
5. Repeat previous steps to add more elements to the configuration.
6. Click Actions, Save Config Only to save the configuration changes.
49
Several zone configurations can reside on a switch at once; however, only one zone
configuration can be enabled at a time.
When enabling a zone configuration from Web Tools, keep in mind that the entire zoning
database is automatically saved and the selected zone configuration is enabled.
If the zoning database size exceeds the maximum allowed, the zone configuration cannot be
enable. The zoning database summary (Zone configuration summary report) displays the
maximum zoning database size. It can be obtained by clicking Print > Print Zone Database
Summary.
To enable a zone configuration:
50
Connectrix Manager offers a way to work with offline zoning. This feature enables you to copy an
active fabric zone DB to edit offline. Some benefits to offline zoning are:
Make changes to the zone database now, but apply them later.
Keep multiple copies of the zone database and switch between them.
Analyze the impact of changes to storage access before applying the changes.
The naming rules for zone names and zone configuration names vary with the type of fabric. Some
rules that apply to FC fabrics are:
Zone names cannot begin with SANav_, SMP_, red_, or default_. Zone set name cannot
begin with redirc_fg. These prefixes are reserved. Any EOS switch with a zone beginning
with a reservered prefix will not be discovered.
Access the Zoning feature from the main screen of the Connectrix Manager application in any of the
following ways:
51
The Zoning Database keeps all zones sets and zone definitions. The Zone DB view consists of
three panels:
Create Zones: The Potential Members panel displays available products and the active
ports that can be put into zones. The zoning method (World Wide Name or Port) is
selected by changing the Zoning Method at the bottom of the column.
Create Zone Sets: The Zones panel shows zones that have been configured. New zones
are added by selecting the New Zone. New members can be added to a zone by
selecting the New Member.
Activate / Deactivate Zone Set: The Zone Configs panel displays zone sets that have
been configured. New zone sets can be configured by selecting New Config. Zone sets
can be exported or imported by choosing Export and Import.
52
A default zone is a zone that contains all products that are not members of a separate active
zone. Each member of the default zone can communicate with every other member of the
default zone.
If default zoning is not enabled for a particular vendor or zoning method, the Default Zone
button is disabled.
EMC does not support enabling the Default Zone.
53
54
55
When a zone configuration is initiated, a number of checks are performed before the Activate Zone Config
dialog box is displayed, and look for the following problems:
Zone and zone configuration name violations
Total zone and zone member limit violations
Zoning configuration violations
Only the selected zone configurations data is sent to the fabric; zone databases are never sent to the fabric.
To activate a zone configuration, complete the following steps.
1. Select the zone configuration you want to activate in the Zone Configs list.
2. Click Activate. Note the following events that may occur:
1. Depending on the characters included in the name of the zone configuration, a message may
display informing that the name contains characters that are not accepted by some switch
vendors.
2. When the total number of zones and zone members defined exceeds the limit recommended for
the system firmware, a warning message displays informing of this fact and asking whether you
want to proceed. Consider carefully whether you want to continue with the zone configuration
activation. The limits are set to ensure stable fabrics; if you proceed, you may undermine the
stability of your fabric.
3. Review the information in the Activate Zone Config dialog box and make sure the selected zone
configuration is the right. Also, select or clear the Generate a report and Store a Copy checkboxes as
desired.
4. Click OK to activate the zone configuration. A message box displays informing you that the zones and
zone configurations you change will be saved in the zone database. When it succeeds, icons for the
active zone configuration and its zones display green. When it fails, the message includes the reason
for the failure.
5. Click OK to continue. The Activate Zone Config dialog box is closed and the Zone DB tab displays.
6. Click OK.
56
The Traffic Isolation Zoning feature allows you to control the flow of interswitch traffic by
creating a dedicated path for traffic flowing from a specific set of source ports (N_Ports). For
example, you might use Traffic Isolation Zoning for the following scenarios:
To dedicate an ISL to high priority, host-to-target traffic.
To force high volume, low priority traffic onto a given ISL to limit the effect on the fabric of
this high traffic pattern.
To ensure that requests and responses of FCIP-based applications such as tape pipelining
use the same VE_Port tunnel across a metaSAN.
Traffic Isolation Zoning does not require a license.
Traffic isolation is implemented using a special zone, called a Traffic Isolation zone (TI zone).
A TI zone indicates the set of N_Ports and E_Ports to be used for a specific traffic flow. When
a TI zone is activated, the fabric attempts to isolate all inter-switch traffic entering from a
member of the zone to only those E_Ports that have been included in the zone. The fabric
also attempts to exclude traffic not in the TI zone from using E_Ports within that TI zone.
57
This lab covers the creation of Virtual Fabrics. Once the Virtual Fabrics are created the lab
covers how to zone B-Series switches in CLI, Web Tools, and Connectrix Manager.
58
This lesson covered zoning tasks in B-Series CLI, Web Tools and Connectrix Manager CNE.
59
This lesson covers MDS-Series virtual SANs. We discuss their use, how to create and
configure them.
60
MDS-Series switches allow one physical switch to be carved into several virtual fabrics called VSANs
(Virtual SANS). VSANs have the following features:
Physical Topology - A physical topology may be partitioned into one or more logical fabrics
called VSANs. Replication of fabric services on a per VSAN basis provides increased scalability
and availability.
Zoning - Zoning is assigned per VSAN. Each VSAN (except the isolated VSAN) may have one
active zoneset configured at a time.
Ports - VSAN membership on the switch is assigned on a port-by-port basis. By default each
port belongs to the default VSAN. VSAN membership can be edited through the CLI or GUI.
Default VSAN - The factory settings for switches in the MDS-Series have only the default
VSAN (VSAN 1) enabled and all ports are assigned to the default VSAN. If more than one
VSAN is not needed, use this default VSAN as the implicit parameter during configuration.
If no VSANs are configured, all devices in the fabric are considered part of the default
VSAN. Note that VSAN 1 cannot be deleted. It can, however, be suspended.
Isolated VSAN - VSAN 4094 is the isolated VSAN. All non-trunking ports are transferred to
this VSAN when the VSAN to which they belong is deleted. This avoids an implicit transfer
of ports to the default VSAN or to another configured VSAN. All ports in the deleted
VSAN are isolated (disabled). Ports may be configured into VSAN 4094. However, when a
port is moved to VSAN 4094, that port is immediately isolated.
When creating VSANs, multiple switches are not needed. A VSAN can be created on one switch.
VSANs can be created using the CLI, DCNM-SAN or Device Manager. The major difference is that with
the DCNM GUI is that VSAN can be created on all switches in one step. With the CLI or Device
Manager, VSANs must be created in each switch separately.
61
The VSAN feature provides two primary functions, hardware-based isolation of traffic, and
independent fabric services for each VSAN:
Hardware-based isolation of tagged traffic belonging to different VSANs requires no special
drivers or configuration at the end nodes, such as hosts, disks, and so on.
Traffic is tagged at the Fibre Channel ingress port (Fx port) and carried across enhanced InterSwitch Links (EISLs) between MDS-Series switches. Because VSANs use explicit frame tagging,
they can be extended over the metro or WAN. The MDS-Series IP storage service module can
add tags to be transported in Fibre Channel over Internet Protocol (FCIP) for greater
distances.
FC, and therefore VSANs, can easily be carried across dark fiber. However, VSANs add 8 bytes
of header, which might be a concern for channel extenders. The channel extenders might
consider it an invalid frame and drop it.
Dense wavelength division multiplexing (DWDM) switches might also count frames as invalid
but might pass the frames anyway.
Each FC fabric service maintains a separate database for each newly created VSAN. These
services include zone server, name server, management server, and principal switch
selection. Each service runs independently on each VSAN and is independently managed and
configured as well.
62
Each VSAN has its own principal switch and domain ID allocation policy, either static or
dynamic. Principal switches for different VSANs do not need to reside on the same physical
switch. Each switch has a separate domain ID for each active VSAN. These domain IDs can
overlap between VSANs unless using inter-VSAN routing (IVR), because IVR requires unique
domain IDs across all switches and VSANs participating in IVR:
Each VSAN can also have a separate Fibre Channel ID (FCID) allocation policy, either static or
dynamic. All ports are originally configured in default VSAN 1. As shown in the figure, each
switch that has end ports in a particular VSAN has a domain ID assigned to that particular
VSAN. Core switches that trunk these VSANs also have assigned domain IDs in these VSANs.
63
For the hierarchical relationship, you first assign physical ports to VSANs. Then you configure
independent zones for each VSAN.
VSANs divide the physical infrastructure, provide traffic statistics, and are changed only when
ports are needed for each VSAN fabric. Zones divide the VSAN infrastructure while providing
added security and allowing the sharing of device ports. Zones can be changed frequently, as
with backup applications where a host might not communicate to a server during the day but
needs to connect to it at night for backup storage. VSANs also allow ports to be added or
removed non disruptively. Zone A on VSAN10 is different and separate from Zone A in
VSAN30.
64
65
66
There are certain rules that must be followed when creating VSANs. VSAN 1, for instance, is
automatically configured by the switch as the default VSAN. All ports that are configured are
originally put into VSAN 1 until specifically configured into another VSAN number. The VSAN
numbers ranging from 2 through 4093 are the user-configurable VSANs. Although there are
more number possibilities in this range, a maximum of 254 VSANs can be created here.
VSAN 4094 is a reserved special VSAN called the isolated VSAN. It is used to temporarily
isolate the ports whose VSAN has been deleted. VSAN 4094 is not propagated across
switches, is always present, and can not be deleted.
Note: VSAN 0, 4095, and 4079 are also reserved and not used.
67
VSAN ID: Identifies the VSAN number. VSAN 1 is set as default and other user-defined
VSANs range from VSAN 2 to 4093. There is also an isolated VSAN 4094.
VSAN name: A text string that identifies the VSAN for management purposes. The
name can be from 1 to 32 characters long and it must be unique across all VSANs. By
default, the VSAN name is a concatenation of VSAN and a four-digit string
representing the VSAN ID. For example, the default name for VSAN 3 is VSAN0003.
Note that a VSAN name must be unique.
VSAN state: The administrative state of a VSAN that can be set to an active (default)
or suspended state. After VSANs are created, they might exist in various conditions or
states. The active state of a VSAN indicates that the VSAN is configured and enabled.
By enabling a VSAN, you activate the services for that VSAN.
Suspended state: Indicates that the VSAN is configured but not enabled. If a port is
configured in this VSAN, it is disabled. Use this state to deactivate a VSAN without
losing that VSANs configuration. All ports in a suspended VSAN are disabled. By
suspending a VSAN, you can pre-configure all the VSAN parameters for the whole
fabric and activate the VSAN immediately.
A VSAN is in the operational state if the VSAN is active and at least one port is up.
This state indicates that traffic can pass through this VSAN. This state can not be
configured.
68
This example shows how to use the CLI to create a VSAN on one switch. If the VSAN includes
multiple switches, then this command sequence must be repeated for each switch.
Explanation of commands:
vsan 10 if VSAN 10 does not exist, this command will create it (with default
attributes)
69
To assign switch ports to a VSAN, they must be added to the VSAN membership list in the
VSAN database. Follow these steps to assign VSAN membership:
70
1.
2.
3.
4.
5.
6.
7.
8.
Note: DCNM-SAN allows us to create the VSAN on several switches with one click of the
mouse. When creating a VSAN with Device Manager or the CLI, a separate operation is
needed for each switch.
71
VSANs can also be created within Device Manager. To do so, open Device Manager and from
the menu select FC, then VSAN.
The VSAN window opens. From it you can see all your existing VSANs and their configuration.
There is a tab that also allows you to see the VSAN membership. Click Create and the Create
VSAN General window opens. Set your parameters and click Create again to complete the
VSAN creation.
72
73
This lesson covered MDS-Series virtual SANs. We discuss their use, how to create and
configure them.
74
This lesson covers zoning tasks in MDS-Series CLI and Data Center Network Manager (DCNMSAN).
75
Use zoning to set up access control between storage devices or user groups. Creating zones
allow to increase network security and to prevent data loss or corruption. It is possible with
administrator privileges in the fabric.
A zone consists of multiple zone members. Members in a zone can access each other. If
zoning is not activated, all devices are members of the default zone. Devices can belong to
more than one zone.
A zone set consists of one or more zones. A zone set can be activated or deactivated as a
single entity across all switches in the fabric. Only one zone set can be activated at any time.
A zone can be a member of more than one zone set. Zoning can be administered from any
switch in the fabric
Zoning in the MDS-Series switches use either CLI or DCNM-SAN. You can not configure zoning
from Device Manager.
76
Each member of a fabric (in effect a device attached to an Nx port) can belong to any zone. If
a member is not part of any active zone, it is considered to be part of a default zone. If no
Zone Set is active in the fabric, all devices are considered to be in the default zone. Even
though a member can belong to multiple zones, a member that is part of the default zone
cannot be part of any other zone. The switch determines whether a port is a member of the
default zone when the attached port comes up. Unlike configured zones, default zone
information is not distributed to the other switches in the fabric. Traffic can be permitted or
denied to members of the default zone. This information is not distributed to all switches; it
must be performed for each switch.
If you change the default zone policy on one switch in a fabric, be sure to change it on all the
other switches in the fabric. The default zone members are explicitly listed only when the
default policy is configured as permit. When the default policy is configured as deny, the
members of this zone are not explicitly enumerated when you issue the show zoneset active
command.
Note: permit is not supported by EMC for normal SAN operation.
77
78
To create a Zone Set and add zones that were previously defined create the Zone Set, give it
a name, and assign it to a VSAN. When creating a Zone Set, you must be in config mode as
indicated by the prompt. The format of the command can be seen above
For the next step, you must be in config zoneset mode, add zones to your zoneset.
Note: If the specified zone name was not previously configured, this command returns the
zone not present error message.
To create zones and add members to the zone from the zoneset prompt, first create a
zoneset, then within the zoneset sub-configuration mode create a zone and add its
members. This zone gets added automatically to the zoneset.
Activate the zoneset so it can be enforced. Copy running configuration to startup
configuration to make this configuration persistent.
79
To view any zone information use the show command. If not any specific information is
requested (for example, a specific zone, zone set, VSAN, or alias, or keywords such as brief or
active), all available information is displayed
To see what zoning information is currently active in the fabric, type
show zoneset.
To display zone information for All VSAN, type
show zone
To display members of a zone, type
80
The default settings for default zone configurations can be changed. Unlike configured zones,
default zone information is not distributed to the other switches in the fabric. When the
switch is initialized for the first time no zones are configured. All members are considered to
be part of the default zone and are not permitted to talk to each other.
To disable the default zone set in the logical domains pane click All VSANs, then select the
Default Zone Policy tab. Click the Default Zone Behavior drop down for the desired switch
and click Deny.
81
Zones are configured within VSANs, however, it is possible configure zones without
configuring any VSANs by configuring them within the default VSAN (VSAN 1). The
VSANs/Zones tab displays the VSANs configured in the currently discovered fabric.
The zones can be changed from the Edit VSAN Local Zone Database dialog. Using the DCNMSAN, right-click a VSAN Zone or Zone Set folder. From the dropdown menu select Edit Local
Zone Database.
82
2.
3.
4.
5.
6. Click OK.
The new zone is displayed in the Database window.
83
3. From this dialog, specify End Devices (host hba and storage devices) or Switch Ports
as members of a zone. Type in the Nx_Port WWN or click to bring up a list of
ports that are logged in to the Name Server. If using the list, select the desired port
and click OK.
84
Zones provide a mechanism for specifying access control, while zone sets are a grouping of
zones to enforce access control in the fabric. If two zone set exist, either zone set can be
activated (but not together).
Changes to a zone set do not take effect in a full zone set until it is activated.
To create additional Zone Sets using DCNM-SAN:
2. Right-click a ZoneSets folder in the left pane, and click Insert from the drop down
menu. The Zoneset Name dialog displays.
85
2.
3.
4.
5.
86
Once zones and Zone Sets have been created and populated with members, it must be
activated.
If zoning is activated any member that is not assigned to an active zone belongs to
the default zone.
1.
2.
3.
4.
5.
Click Yes to review the differences. The Local vs. Active Differences dialog box open
Click Close to close the dialog box. The Save Configuration dialog box open.
Check the Save Running to Startup Configuration check box to save all changes to
the startup configuration.
6. Click Continue Activation to activate the zone set, or click Cancel to close. The dialog
box and discard any unsaved changes.
7. The Zone Log dialog box open, which shows if the zone set activation was successful.
If one Zone Set is active and you activate another Zone Set, the currently active Zone Set is
deactivated.
87
This lab covers the creation of Virtual SANs. Once Virtual SANs are created the lab covers
how to zone MDS-Series switches in CLI and Data Center Network Manager (DCNM-SAN).
88
This lesson covered zoning tasks in MDS-Series CLI and Data Center Network Manager
(DCNM-SAN).
89
This lesson covers the steps required to provision storage from a CLARiiON to a host with
Navisphere Manager.
90
This is a roadmap to configuring a CLARiiON. We won't deal with the initialization of the array
since it is assumed the array has already been setup and can be accessed using IP. These
tasks can be performed through Navisphere Manager or CLI.
91
CLARiiON arrays can be managed from any host using Navisphere Manager as long as it has
an out-of-band, IP connection to the CLARiiON.
Navisphere can manage up to 100 storage systems in a CLARiiON domain. It allows easy
configuration of storage objects such as RAID Groups, LUNs, metaLUNs and Storage Groups.
The LUN folders option helps to organize LUNS and other user defined objects specially in
larger environments.
Common functions are performed through the Task Bar Wizard such as LUN creation, Mirror
creation and SnapView operations. LUNS are easily assigned to hosts, or re-assigned to other
hosts by inclusion in Storage Groups.
Any storage system errors that should occur are immediately highlighted on the GUI by a
change in the affected objects icon. The administrator may then take the necessary steps to
correct the problem.
In addition to allowing easy management of storage system configuration and basic storage
objects, Navisphere Manager also allows storage system replication software such as
MirrorView, SAN Copy and SnapView to be managed from the same interface.
To access the Navisphere Manager application, type SP-As IP address into an internet
browser.
92
The Task Bar menu can be used to simplify the creation of RAID Groups. By selecting the
Provision (previously Allocate) icon from the Task Bar Window, you can launch the Wizard.
From the Wizard, you can select a server, storage array, change different LUN parameters,
create (previously bind) LUNs, and create Storage Pools which include both RAID Groups and
Thin Pools.
93
From the Welcome Screen, click Next. Note the radio button for Assign LUNs to a server is
selected by default. You may choose to assign the LUNS to a server at this point, or if you
want to allocate the LUNS at another time, select Continue without assigning LUNs at this
time.
The example shows the Select Servers screen, where w3k_241 is selected. Click Next.
94
Available storage systems are shown in the Select Storage System screen. Highlight an array
and then click Next.
95
The Select LUN Type screen gives the user the option of creating either a Traditional LUN or a
Thin LUN. After selecting Traditional, the user then creates a RAID group. If the user selects
Thin he is then prompted to create a Thin Pool (covered later). Select Traditional and click
Next.
96
Available RAID Groups are shown in the Select Storage Pool screen. By clicking the Storage
Pool Properties, you can view the RG parameters of the group.
A new RAID Group (for a traditional LUN, or FLARE LUN abbreviated FLU) can be created by
clicking the New Storage Pool button.
If the Storage Pool is already created, just highlight the Storage Pool, then click Next.
97
This screen allows the configuration of the Storage Pools LUN properties. The Automatically
assign LUN IDs as LUN names is the default. LUNs will be numbered in order of the next
available lowest number (e.g. 0,1,2,3,4,5,6 etc). You may choose to create more than a single
LUN and supply a name and sequence number as well. For example, creating 5 LUNs with the
name DB_LUNs and sequence number of 1would configure the following: DB_LUNs_1,
DB_LUNs_2, DB_LUNs_3, DB_LUNs_4, DB_LUNs_5.
The RAID Type displays the RAID type previously assigned and the User capacity assigns the
capacity for each LUN.
98
The Select Folder screen gives the user the ability to assign a newly created LUN to a specific
folder for organizational purposes. When click Next the Summary window appears. If some
parameter needs to be changed, select the Back option. Otherwise, click Finish.
99
Navisphere Manager allows the use of Thin LUN provisioning. Thin LUN provisioning delivers
an adaptive means of presenting array capacity to the host by growing as more space is
requested. This improves the utilization of physical capacity and allows a greater ease of
provisioning. In the Storage Provisioning Wizard, select Thin and click Next.
100
Available Storage Pools are shown in the Select Storage Pool screen. By clicking Storage Pool
Properties, you can view the Thin Pool parameters.
A new Thin Pool can be created by clicking the New Storage Pool.
If the Thin Pool is already created, just highlight the Thin Pool, then click Next.
101
From the LUN properties screen you choose the number of LUNs to create, the LUN Name,
sequence number, and the user capacity.
102
The select folder menu gives the option of adding LUNs to a specific folder. Click Next. The
Summary window appears. If some parameter needs to be changed, select the Back option.
Otherwise, click Finish.
103
The next few slides guide you through the steps needed to create Traditional LUNs and Thin
LUNs in the Enterprise Storage window. Once created, you can view and change LUN
properties, create, expand, and destroy Thin LUNs, and finally destroy LUNs. The list on the
slide details the key points for managing LUNs.
104
The General tab in the Create LUN window displays the Storage Pool Properties, Capacity,
and LUN properties.
Storage Pool Properties
Storage Pool Type - Either RAID Group or Thin Pool
RAID Type - Sets the RAID type of the LUN you are creating
Storage Pool (previously RAID Groups) for new LUN - Sets the Storage Pool for the
LUN you are creating. It displays only those Storage Pools that contain the proper
number of disk for the selected RAID type. The RAID Group assumes the RAID type of
the first LUN that is created within it. The RAID Group IDs range from 0 through 239;
the RAID Group ID is assigned when the RAID Group is created
Capacity
Available Capacity vs. Consumed Capacity - The amount of capacity of the Storage
Pool available for creating LUNs vs how much is being used.
Largest Contiguous Free Space - Largest contiguous span of free space in the Storage
Pool. LUNs must fit into a contiguous span of free space
LUN Properties
User Capacity Size of LUN you want to create. Allows the user to select the size of
the LUN in MBs, GBs, TBs or Block count.
LUN ID - Sets the LUN ID of the new LUN. The default value is the smallest available ID
for the currently selected storage system. Highlight the box and enter the LUN ID or
use the drop down arrow.
105
Enable Read / Write Cache Both are enabled by default. Generally, you should enable read
caching for every RAID type that supports it. If you want faster read performance on some
LUNs than on others, you may want to disable read caching for the lower priority LUNs.
Enable Auto Assign Auto assignment (disabled by default) controls the ownership of the
LUN when an SP fails in a storage system with two SPs. With auto assignment enabled, if the
SP that owns the LUN fails and the server tries to access that LUN through the second SP, the
second SP assumes ownership of the LUN so the access can occur. If you are running failover
software such as PowerPath on a server connected to the storage system, you must disable
auto assignment for all LUNs that you want PowerPath to fail over when an SP fails.
Rebuild Priority Sets the rebuild priority for the rebuild operations that occur automatically
with a hot spare and after you replace a failed disk.
Verify Priority The verify priority defines the relative importance of checking parity sectors
in a LUN. If an SP detects parity inconsistencies, it starts a background process to check all
the parity sectors in the LUN.
Alignment Offset - If available, can be used when the host operating system records private
information at the start of the LUN. The default value is zero and this supports most host
operating systems.
No Initial Verify Performs an initial background verify on the newly created LUN, and
eliminates any latent soft media errors.
Default Owner - The default owner is the SP that assumes ownership of the LUN when the
storage system is powered up. If the storage system has two SPs, you can choose to create
some LUNs using one SP as the default owner and others using the other SP as the default
owner.
106
A user can create LUNs for a Thin Pool by right-clicking a Thin Pool in the Enterprise Storage
window and selecting Create LUN.
When creating a Thin LUN, there is a General and Advanced menu. The menus are similar to
that of creating a traditional LUN except for the Oversubscribed by detail in Capacity and the
smaller amount of Advanced options.
107
The next few slides take you through the steps necessary to manage Storage Groups. Once
created, you can view and change storage group properties, add and remove LUNs, connect
and disconnect hosts, and destroy the Storage Group. The list on the slide details key
components of managing storage groups.
108
By right-clicking the Storage Group icon, you can create Storage Groups on a shared storage
system, as long as Data Access control is enabled for the storage system.
Storage System - Name of the storage system on which you want to add a new
Storage Group
Storage Group Name - Name of the Storage Group you are creating. Displays the
default name for a new Storage Group. The name Backup has been typed in in the
example.
When creating a Storage Group, the software requires only a name for the Storage Group. All
other configurations are performed after the Storage Group is created. As part of the Wizard,
you can have a storage group automatically created.
109
The General tab displays general properties about the Storage Group. It displays the current
properties of the selected Storage Group and lets you modify some of these properties.
Storage System - Name of the storage system that owns this Storage Group.
Storage Group WWN - World Wide Name (WWN) assigned to this Storage Group
Storage Group Name - Name of the Storage Group. You can enter a new Storage
Group name here.
110
The LUNs tab is used to add or remove LUNs from a storage group or verify which are
members.
The Show LUNs option allows the user to choose whether to only show LUNs which are not
yet members of any storage group, or to show all LUNs. The latter allows the user to place
LUNs in multiple Storage Groups, and thereby share LUNs between hosts. LUNs may be
added to, or removed from, storage groups by selecting them in the appropriate pane.
LUNs are grouped under the LUN Folder on the storage system.
At the time a LUN is added, the user may select the host LUN ID by scrolling to the right in
the Selected LUNs pane, highlighting the LUN, clicking the Host ID field, and choosing the
host ID from the dropdown list.
111
The Hosts tab allows the user to add or remove hosts in much the same way as LUNs. The
tab displays the hostname, operating system, and host IP address. Highlight the host and use
the arrow to move it to the right side to connect the host to the storage group.
112
The CLARiiON supports integration with the VMware ESX Server. This gives the CLARiiON the
ability to maintain end to end mappings for LUNs with virtual machines. Virtual Integration
Logic on a CLARiiON array will contact ESX server Web Services over IP through VMware VI
SDK to gather information. The host tab in the Enterprise Storage window can now be used
to see which virtual machines are attached to each host.
113
From the LUN Properties Host tab a user can view the Virtual Machine and ESX server to
which a LUN is mapped. The information displayed here includes the IP address, device
mapping, and device name of the ESX Server and the name, type disk mode, disk capacity,
and file path of the Virtual Machine.
114
From the Host Properties, Virtual Machines tab, a user can view information of the Virtual
machines connected to the specified storage group. This includes the VMs name and the
guest host name, IP address, and operating system.
115
The Storage tab in the Virtual Machines properties tab shows all the LUN mappings in use by
the VM. LUN Mapping shows all LUNs in use by VM. VM Info shows VM devices, types, and
the LUNs used.
116
This lab covers the provisioning of LUNs to Hosts using Navisphere Manager.
117
This lesson covered the steps required to provision storage from a CLARiiON to a host with
Navisphere Manager.
118
119
Unisphere is web-based software that allows you to configure, administer, and monitor VNX series. It
provides the user with an overall view of what is happening in your environment plus an intuitive and
easier way to manage EMC unified storage. After selecting a system the storage system dropdown
lists the Dashboard view is presented to the user. Here users can view system alerts, system
information, and the storage capacity summary of the current array.
From the Unisphere System view, users can select either the Hardware or Monitoring. The Hardware
option allows the user to configure, view, and service the systems hardware components. The
Monitoring and Alerts option allows the user to monitor the system health and configure
notifications for important events.
From the Unisphere Storage view, users can select Shared Folders, LUNs, Virtual Tapes, Data
Migration, and Storage Configuration options. The option Shared Folders is used to create and
manage CIFS shares and NFS exports for file. When the users select LUNs on this page they are
allowed to create and manage LUNs for block.
The Virtual Tape option allows the users to create and manage storage that emulates physical tape
devices. Data migration option is used to create and manage the file systems migrations and SAN
Copy sessions.
Finally the option Storage Configuration allows for the creation and management of file systems,
storage pools, and volumes. The Hosts tab displays the hosts you want to connect to the Storage
Group. The Data Protection view allows the user to manage their replication technologies. Wizards
for Snapshots, Clones, and Mirrors are provided here.
The Settings view has the option allows the management of Network, Security, and Data Mover
Parameters.
The Support view provides the user with resources to help the user manage Unisphere. These include
How To procedures, Unisphere Help, Community resources, Product Support pages, Downloads,
and EMC support.
120
We begin our lesson by viewing the Host Properties to see what storage is assigned at this
time to a specific host. In our example, there is no storage group assigned to the host and
consequently there are no LUNs assigned either.
121
You can view the Storage Pool Properties to see if there is adequate storage to assign to your
host based on your servers needs. In our example, we have been asked to build a separate
Pool for our host. However, for the sake of our demonstration we will showi the wizards for
building both a Pool & RAID Group storage.
A Pool is a set of disks (minimum two; maximum depends on storage system), all with the
same redundancy (RAID 1/0, RAID 5, or RAID 6 only), on which you create one or more LUNs
with or without the thin property. For more efficient performance, we recommend that all
disks have the same capacity.
A RAID Group is a set of disks (minimum 1; maximum 16), all with the same RAID type, on
which you create one or more LUN or metaLUNs. For more efficient performance, we
recommend that all disks have the same capacity and speed.
A pool is somewhat analogous to a RAID group , which is a physical collection of disks on
which logical units (LUNs) are created. Pools are dedicated for use by pool (thin and thick)
LUNs. Pools can contain a few disks or hundreds of disks, whereas RAID groups are limited to
16 disks. Because of the large number of disks supported in a pool, pool- based provisioning
provides the same benefits of a metaLUN being striped across many drives, and, unlike
MetaLUNs, it requires little to no planning and/or management effort.
Pools, like RAID groups, support a single RAID protection level which can be RAID 5, RAID 6
or RAID 1/0. Pools can be homogenous (having a single drive type) or heterogeneous
(containing different drive types).
122
Clicking the LUN Provisioning Wizard link launches the wizard so that you may assign LUNs to
your host. This wizard takes you through the steps to create Storage Pools, if needed and
thereafter LUNs to assign to your servers.
You need the following for this wizard to run properly:
A Server to assign the LUNs to. This step is optional since there are other reasons that
you might be creating LUNs; such as LUNs for the Reserved LUN Pool.
The Size and the Parameters for the LUNs you wish to create.
123
We select the Server that you wish to assign LUNs to. In our example, we are creating storage
for use by a Windows 2008 server. It is also possible to assign storage to more then one
server using this wizard.
If the server is already in a storage group, then the LUNs will be added to the existing storage
group. If a server is not in a storage group then a new storage group will be created for the
server and the LUNs will be placed in it. The naming convention for the newly created
storage group will be SG_<server name>. So in our example, the name is SG_Kramer.
Click Next when ready.
124
Select the Storage System that you wish to assign LUNs from. It is possible to only select one
system at a time. Storage systems that are not in a local domain are not listed.
Click Next when ready.
125
Select the Storage Pool that you wish to assign LUNs from. As previously mentioned, a
Storage Pool can consist of a Pool or a RAID Group. If you wish, then you can also build a new
Storage Pool by clicking the wizard New Storage Pool as shown here.
For the sake of our demonstration we show both the creation of a Pool and a RAID Group.
126
Create the Pool or RAID Group that you wish to create LUNs from. In our example we will
choose Pool.
Some of the Storage Pool Parameters are as follows:
The Storage Pool Type specifies the type of storage pool to create, either Pool or
RAID Group.
The Scheduled Auto-Tiering is visible only when the FAST enabler is installed and the
Pool storage pool type is selected. Select Scheduled Auto-Tiering to include this
storage pool in the auto-tiering schedule. The auto-tiering state is set to Scheduled. If,
at a later time, you clear Scheduled Auto-Tiering, the pool will no longer participate in
the auto-tiering schedule and the auto-tiering state will be set to Manual. You can set
or unset Scheduled Auto-Tiering later.
The Storage Pool ID is for Pool or RAID group ID number. For pools, the value is the
smallest available ID for the currently selected storage system, and it is not editable.
For RAID groups, the default value is the smallest available ID for the currently
selected storage system. You can assign a different ID from the Storage Pool ID list.
The Storage Pool Name is for pools, a software-assigned name that is editable and
must be unique across all pools.
And the RAID Type: For pools, shows only RAID 6, RAID 5, and RAID 1/0. There must
be enough disks available to support the RAID type.
127
Storage Pool terminology is listed here. Please take a moment to review these definitions.
128
Storage Pool terminology is continued on this page. Please take a moment to review these
definitions.
For more information on these terms please see the online help.
129
Advanced options allow you to set the Pool description, Gauge the Pool Alert Threshold and
enable Fast Cache capabilities. Some of the Pool Advanced Parameters are as follows:
The Pool Description allows you to enter an identifying description for the pool.
The Pool Alerts: allows you to set the Percent Full Threshold. This is the consumed
capacity of the pool that triggers an alert. The default value is 70%. We recommend
that you set the value somewhere between 50 and 75%. Regardless of the value that
you set, the storage system automatically triggers an alert when the consumed
capacity reaches the 85% full threshold value and, rather than trigger a new alert,
updates the alert with each 1% increase in consumed capacity.
And when the FAST Cache is selected, it enables the FAST Cache. It only displays if the
FAST Cache enabler is installed.
130
Storage Pool ID: Pool or RAID group ID number. For RAID groups, the default value is
the smallest available ID for the currently selected storage system. You can assign a
different ID from the Storage Pool ID list.
Storage Pool Name: For RAID groups, a software assigned name (RAID Group x) that
is not editable.
RAID Type: For RAID Groups, it shows all RAID types supported for the storage
system. Again, there must be enough disks available to support the RAID type. RAID 5
is the default RAID type for Pool and RAID Group.
And as before you have the ability to allow the system to automatically choose the disks for
you or to choose them manually.
131
Automatically delete after last LUN is deleted: When this choice is selected, it makes
sure the system destroys this RAID group after all LUNs belonging to this RAID group
are deleted (or destroyed). When cleared, the RAID group is not destroyed.
RAID Group Power Settings: Allow Power Savings enables power savings for the
RAID group.
For our demonstration, we stay with using a Pool to build and assign our LUNs from.
It should be noted that whether or not you can take advantage of the Power Saving is
dependant upon your drive type choice. The NL-SAS drives can spin down, but not SAS
drives. Also Flash Drives by their very nature do not spin down so they are also not eligible
for this feature.
132
The new storage pool we created, Pool 1, is now available and we can now proceed with the
wizard to decide on what type of LUNs to have in our pool.
Choose Next when ready.
133
We select LUN features next. The Virtual Pools feature allows you to allocate storage using thick and
thin LUNs within pools. Thick LUNs and thin LUNs can reside within the same pool and they can share
the pool's storage capacity. Both thick and thin LUNs can be easily provisioned, expanded, and
compressed via the Unisphere interface.
A thin LUN lets you assign more storage capacity to a host than is physically available. Storage is
assigned to the server in a capacity-on-demand method from a shared pool. A thin LUN competes
with other LUNs in the pool for the available pool storage. The storage system software monitors and
adds storage capacity, as required, to each pool, not each LUN. This simplifies the creation and
allocation of storage capacity. For thin LUNs, you must install the thin provisioning enabler on the
system.
The actual consumed capacity and rate of consumption for the thin LUN can vary depending on the
attached host file system or the application using the LUN. This is a normal condition typical of most
thin provisioning services.
Thick and Thin LUNs are supported for all VNX systems and are part of the base code. The main
difference between a Thick and Thin LUN is that a Thick LUN reserves all of its assigned capacity at
the time it is created.
A few rules to remember when working with virtual LUNs are:
Virtual Pools are supported for MirrorView/A, MirrorView/S or SAN Copy replication
software.
Thin LUNs cannot be used in the reserved LUN pool, the Write Intent Log, or as a clone
private LUN.
Thick LUNs cannot be used in the Write Intent Log, or as a clone private LUN. They can be
included in the reserved LUN pool.
Thin and thick LUNs cannot be a component in a metaLUN.
Virtual Pools support RAID 5, RAID 6 and RAID 1/0 types and RAID 5 is the default RAID type.
Pools do not support the hot spare RAID type.
Pools support Flash drives, SAS, and NL-SAS drives.
Choose Next when ready.
Copyright 2013 EMC Corporation. All rights reserved
134
On the LUN Properties screen you have the ability to choose the number of LUNs you want
to create as well as the User Capacity. Because we are choosing to build Thin LUNs not all of
the User Capacity will be pre-allocated. Rather it will be used on an on-demand basis as
previously mentioned.
We also have the ability to assign a LUN Name Format. This is a user-assigned, nice name,
that is easily recognizable as to the purpose of the LUNs. In our demonstration, we have
chosen Kramer with a concatenated Starting ID of 0. So our first LUN will be called Kramer_0
and so on.
Choose Next when ready.
135
Optionally we can assign our LUNs to folders. The folders feature allows you to organize
your storage system LUNs for optimum usability and efficiency. You can create folders and
then place LUNs, metaLUNs, and snapshots in the folder. Once you define folders and assign
LUNs to them, the LUNs are organized within the folders and displayed in the Unisphere User
Interface.
The folders node includes two system-defined folders - SP A and SP B. Each folder includes all
LUNs assigned to it when the LUN was bound as shown in the graphic on the right.
For our demonstration, we will create a new folder called Kramer Host and assign our LUNs
to it.
Choose Next when ready.
136
At our Summary screen we have the ability to review and make alterations to the
instructions (if necessary) and choices we made to the wizard.
Choose Finish when ready.
137
The next screen we see is the Results of the Provisioning Wizard. Here we can see the
step-by-step process the system ran through from the choices we made.
Choose Finish when ready.
138
We revisit the Host Properties and we see the LUNs we created with the naming convention
we chose for them in the wizard. We still need to refresh the LVM of the Windows host in
order to see the new storage devices.
139
In the Logical Volume Manager of the host, after a refresh and rescan, we see our newly
acquired storage. In order for the Host to use this, we need to format and mount.
140
Using the Windows Disk Management, we can initialize, format and mount our drives so they
may be used as block storage by the server and its applications.
141
In the Host Properties, we have the ability to update the information about the host that is
stored on the VNX Array. This option is only available if there is a VNX Host Agent on the host
or it is an ESX/ESXi server. The type of information includes SCSI Device Address, Physical
Device Address, File system on the device and the storage system the device belongs to.
142
Once the update has run through, we see the up-to-date information on the system and the
devices in the details window as shown here.
143
This lab covers the provisioning of LUNs to Hosts using Unisphere Manager.
144
145
This lesson covers the steps required to provision storage from a Symmetrix to a host with
Symmetrix Management Console.
146
Step 1 Creating new hyper volume extensions is considered one class of change. Creation of certain device
types may require more than one class of command execution (for example a RDF protected BCV). The
Symmetrix must have enough unconfigured disk space in order to create new devices.
Step 2 To access a new device from a host system, you need to map the device to one or more front-end
director ports and then update the host and the SYMAPI database. Front-end mapping is a Symmetrix
mechanism for exporting the logical view of a device to a host system. After you map a device, the host is
usually unaware of it until you run a host utility that allows the host to address the new device. To map a
device, use the map command file entry to specify the front-end director number and port number. For FBA
devices specify the logical unit number (LUN) for SCSI or fibre, the target ID for SCSI, the Virtual bus (vbus)
address for mapping to a fibre adapter (FA) port if volume set addressing is being used (for HP-UX) or (if volume
set addressing is not being used) only the LUN. For CKD devices specify the CKD device number (when mapping
a CKD device to an OS/390 host). If also updating a device masking database specify the HBA identifier (WWN,
AWWN, or ISCSI name).
Step 3 Device (LUN) masking ensures that volume access to servers is controlled appropriately. This prevents
unauthorized or accidental use in a distributed environment. A zone set can have multiple host HBAs and a
common storage port. LUN Masking prevents multiple hosts from trying to access the same volume presented
on the common storage port. LUN Masking is a feature offered by EMC Symmetrix and CLARiiON arrays.
When servers log into the switched fabric, the WWNs of their Host Bus Adapters (HBAs) are passed to the
storage fibre adapter ports that are in their respective zones. The storage system records the connection and
builds a filter listing the storage devices (LUNs) available to that WWN, through the storage fibre adapter port.
The HBA port then sends I/O requests directed at a particular LUN to the storage fibre adapter. Each request
includes the identity of their requesting HBA (from which its WWN can be determined) and the identity of the
requested storage device, with its storage fibre adapter and logical unit number (LUN). The storage array
processes requests to verify that the HBA is allowed to access that LUN on the specified port. Any request for a
LUN that an HBA does not have access to returns an error to the server. The VCM database can be unmapped
from any director that is not being used for masking control. If you have PowerPath installed, the VCM should
be mapped.
147
For users of the Symmetrix, Symmetrix Management Console provides intuitive, browserbased device management.
Symmetrix Management Console provides full management control of individual Symmetrix
systems for those environments that do not need advanced Storage Resource Management
capabilities, or for those that simply need a lightweight graphical interface to complement
their SRM infrastructure.
Symmetrix Management Console reduces the complexities associated with a command-line
interface for system management, managing the system more efficiently and effectively. This
helps improve staff productivity and maximize utilization of the system resources, while
reducing access time to the critical business information.
148
149
To perform the creation of FBA Meta Devices. Right-click a Symmetrix and choose FBA Meta
Device Configuration, then choose Form Meta. In the example shown, the device filter is
used to only show those devices reserved by the current SMC user. The Meta Configuration
can be concatenated or striped, in this example we are creating a Concatenated meta device.
Highlight the devices that should form the meta from those listed in the unmapped devices
list and click Add. This moves the devices to the Meta members column. The meta head can
then be specified. As with all configuration tasks, click Add to Config Session List. The actual
commit of this action is done from ConfigSession view.
When creating a meta, you can optionally use the Auto Select feature. This allows you to
specify only the number of metas, number of meta members per meta, and the meta heads;
the Symmetrix microcode automatically chooses the meta members from the available pool
of unmapped devices.
150
Device mapping is done by right-clicking either the Symmetrix, device, or device folder and
choosing Device Masking and Mapping, then Map Devices. This launches the Mapping
Wizard, which is five pages in length, one for each step of the process.
1 - The devices to be mapped are chosen.
2 - The ports to which the devices are to be mapped.
151
152
Starting with Solutions Enabler 7.0 and SMC 7.0, for pre V-Max arrays, a set of host initiators
(HBAs) can be grouped together with an Alias name. In addition, each individual host initiator
port can be aliased as well. In pre V-Max Symmetrix Arrays, the VCM bit must be set on a FA
port for device masking. The VCM bit can be set via SMC or Solutions Enabler.
Aliasing makes the process of device masking easier because one can now use an easy to
remember alias instead of the WWNs. In addition, using the Host Alias name one can
perform masking operations on a number of initiators in one step, instead of masking each
initiator individually. In order to use Aliases, the host initiator records must exist on the
Symmetrix, i.e. the HBA must have logged into the Symmetrix at some point. Usually the
process of zoning an HBA to a VCM enabled front end port results in the HBA logging into the
Symmetrix.
To create a host masking alias, right-click a Symmetrix and choose Device Masking and
Mapping > Create Masking Alias. Enter an alias name, then choose the initiators from the
Available column on the left and add them to the Target column on the right. In this example
we are creating an Alias for an ESX Server so we choose the alias name of ESXServer_178,
then we have added the ESX servers initiator into the target column.
The Host Alias folder is in the Symmetrix Masking folder for each Symmetrix.
153
The masking dialog can be initiated by right-clicking a Symmetrix, Symmetrix port, HBA, or Host Alias
and then choosing Device Masking and Mapping followed by Masking. This launches the masking
dialog shown.
One can choose to mask on a per initiator basis or by using a host alias.
First choose a director port from the dropdown menu. Then pick an initiator (usually a host HBA) or
Host Alias by clicking the radio button. Choose one of the Host Aliases previously created or one of
the initiators that have already logged into the Symmetrix or type in a worldwide name. In the case
of an Open Replicator / SAN Copy environment, the initiator could also be a storage port.
If you are running SMC Version 6.0 or higher, you have the ability to enable dynamic addressing here.
You can check the box to enable dynamic addressing. If you enable this, you can click the Set
Dynamic LUN Address button in the lower left to manually assign host LUN addresses yourself, or just
click OK in the dialog to allow the system to dynamically assign them for you. In the example above,
we are using a Host Alias and the Enable Dynamic Addressing box has not been checked, so the Set
Dynamic LUN Address is greyed out.
Next, add devices from the Available list to the Target list, as shown in the graphic. You can use the
appropriate filter, including the reserved devices filter mentioned earlier, to narrow the devices
shown in the Available Devices column. Unmapped devices can also be included in this list if the box
is checked. Choose one or more devices to be masked and add them to the Target column.
Apply executes the masking operation and keeps the dialog open for further masking operations; OK
executes the masking operation and closes the window. The VCMDB can be refreshed optionally, as
recommended, when you click Apply or OK. Note that masking does not use the Config Session list.
Clicking Apply or OK commits the masking operation immediately.
154
1. VCMDB Maintenance Operations for pre V-Max arrays are performed by right-clicking a
Symmetrix and choosing Device Masking and Mapping.
2. Choose Masking Information Maintenance. This launches the Masking Information
Maintenance dialog.
3. Choose Backup from the Action dropdown for a backup, and then specify a backup file
name and click OK.
4. Choose Restore for a restore operation, then choose a filename from which the restore is
to be performed.
A number of other VCMDB maintenance actions can also be performed from this dialog. The
possible actions are shown on the slide.
155
Autoprovisioning Groups is a feature introduced with the Symmetrix V-Max Series with
Enginuity 5874 and is supported with EMC Solutions Enabler and Symmetrix Management
Console version 7.0 or higher. For V-Max Symmetrix Arrays, the ACLX bit must be set on a FA
port for device masking. The ACLX bit can be set via SMC or Solutions Enabler.
Autoprovisioning is an easier way to manage mapping and masking tasks, has fewer actions
to execute and adds simplicity to mapping and masking Symmetrix logical volumes. This is
accomplished with the creation of three independent groups and then associating those
groups into what is called a Masking View. The three groups are:
Initiator Group
Port Group
Storage Group
These three independent groups are associated into a Masking View.
156
An Initiator Group contains the world wide name or iSCSI name of a host initiator, also
referred to as an HBA or host bus adapter. An initiator group may contain a combination of
up to thirty-two, Fibre Channel initiators or eight, iSCSI names or a combination of both.
There is a limit of 8,192 initiator groups in a Symmetrix V-Max array. Port flags are set on an
initiator group basis, with one set of port flags applying to all initiators in the group.
However, the FCID lockdown is set on a per initiator basis. An individual initiator can only
belong to one Initiator Group. Once the initiator is in a group, the group can be a member in
another initiator group. It can be grouped within a group. This feature is called Cascaded
Initiator Groups and is only allowed to a cascaded level of one. This feature is explained in
more detail later in this module. A Port Group may contain any number of valid front end
ports, FAs. Front end ports may belong to more than one port group.
There is a limit of 512 port groups. There is one restriction that is mentioned here and then
explained in more detail later. Before a port can be added to a port group the ACLX flag must
enabled on the port.
A Storage Group may contain up to 4,096 Symmetrix logical volumes. A logical volume may
belong to more than one storage group. There is a limit of 8,192 storage groups.
157
158
159
160
6. Click the Set Dynamic LUN Address to manually change the host LUN addresses. By
default, the next available host LUN address is automatically assigned to the devices.
7. Click OK to complete the creation of the Masking View.
As soon as the Masking View is created, the host reconfiguration steps can be performed
and the host should see the V-Max devices.
161
SMC allows the creation of storage templates which can be used later when provisioning
storage to V-Max arrays. Storage Templates are available in SMC only.
1. To create a new template, change the SMC view to the Tasks view.
2. Click the Template Wizard Hyperlink.
3. Click Create in the Template Manager window to launch the Template Wizard.
The first step is the Welcome page (not shown on slide). In the second step the name of the
Template and the Symmetrix ID is specified (one can choose Any for the Symmetrix ID).
162
163
Storage Templates can be used when creating a new Storage Group or when expanding an
existing Storage Group.
To expand an existing Storage Group, right-click the existing Storage Group and choose
Device Masking and Mapping, then choose Storage Groups Maintenance and then Expand
Storage Group. Select a storage template that defines the requirements of the storage to
add to the group, or select NA to use existing devices as a model. In this example we have
chosen the template StorageForPayroll. If you specify a storage template, then no other
information is required. Click Next for the summary and then click Finish.
If a template is not used (N/A), then specify the total capacity to create. Specify the method
to use when selecting devices, the emulation to use and the thin pool to use if TDEVs are
being used. One may also have to specify the SSID if Symmetrix array has ESCON or FICON
directors.
164
Masking Information Maintenance Operations for V-Max arrays are performed by rightclicking a Symmetrix and choosing Device Masking and Mapping, then Masking Information
Maintenance. This launches the Masking Information Maintenance dialog.
Choose Backup from the Action dropdown for a backup, and then specify a backup file name
and click OK.
Choose Restore for a restore operation, then choose a filename from which the restore is to
be performed.
165
This lab covers the provisioning of LUNs to Hosts using Symmetrix SMC.
166
This lesson covered the steps required to provision storage from a Symmetrix to a host with
Symmetrix Management Console.
167
This module covered SAN management aspects from host and storage configuration to
connectivity.
168
This module focuses on how to configure multiple switches in a fabric. The module also
covers iSCSI configuration.
This lesson covers an overview of fabric configuration. Build process, fabric entities and path
selection are covered.
If two enabled switches are cabled together, they must have unique Domain Ids, or
the Fabric will be segmented.
Ensuring unique Domain ID
If a new switch is cabled to the Fabric and then enabled, the master switch assigns a
unique Domain ID to the new switch.
This checklist can be used to serve as a tool for completing all steps before actually merging
fabrics.
Verify that each switch has a unique Domain ID. When merging fabrics, ensure that
there are no duplicate Domain IDs among all switches that will be part of the merged
fabric.
Verify that all switches have been set up to work in a supported interop mode.
Verify that the E_D_TOV and R_A_TOV are set the same on all switches that will be
part of the new fabric. (By default, they should all be the same; if necessary, refer to
the appropriate user manual for information on how to set up operating parameters.)
Note: Switches use different units to represent the same values; for example a value
of 2000 on a B-Series switch or MDS-Series switch is the equivalent of 20 on a
Brocade M-Series switch.
Verify that the active zone set has been checked (with the respective switch fabric
management tools) and does not contain illegal characters.
If a switch is not operational and the zoning definition on that switch is not required,
be sure to clear the zoning configuration on that switch.
If a switch is operational and the zoning configuration on that switch is required, be
sure to check that there are no duplicate active zone names. If there are duplicate
zone names, rename one of the zones.
Ensure that all switches are configured with WWN zoning.
Ensure that all switches comply with proper zone naming.
Back up the switch configuration by issuing the appropriate commands.
When we discuss Fabric Configuration, in essence what we are talking about is a Fabric
Merge between two or more switches
1. The port initialization takes place first. In this step, the port determines the Port Type.
For this discussion, we assume the port has configured itself as an E_Port (Expansion
Port). Once this port is a recognized E_Port, Exchange Link Parameters (ELP) are sent.
The ELP exchanges information about the switch such as its WWN and Class of
Service.
2. The Fabric must select a Principal Switch. This is accomplished by the Exchange Fabric
Parameters (EFP) and Principal Switch Selection (DIA) operations.
3. Once the Principal Switch is elected, Domain_IDs are assigned to the other member
switches of the Fabric. Switches attempt to keep their previous Domain_ID.
4. The final steps in configuring the fabric are to build the topology database using
Fabric Shortest Path First (FSPF) and then create the routing tables.
Although not specified, the switches attempt to combine the existing zone information into a
single zone set to be distributed throughout the new Fabric.
A Link Initialization occurs when a cable is plugged into the SAN to determine what port type
were dealing with. Ports within a SAN mainly fall Into one of four types. There are many subtypes but they are all more specific versions of the following port types:
Node (N) Port: Transmits and receives Fibre Channel data frames in a Switched Fabric
environment. N-ports represent end points in the fabric, such as SCSI initiators and SCSI
targets.
Fabric (F) Port: Port on a switch that provides access to fabric services (eg. FLOGI, PLOGI). FPorts connect to N-Ports to form a fabric point-to-point.
Expansion (E) Port: Port on a switch that allows connectivity (expansion) to another switch.
Generic (G) Port: Port on a switch that detects the attached port type and auto-configures to
match that port type.
In this module, we review the E Port and its various sub-types.
Switches are connected to each other in a fabric using Interswitch Links (ISL) using expansion
ports on the switch (E_Port). ISLs are used to transfer host-to-storage data, as well as fabric
management traffic from one switch to another. They are the fundamental building blocks
used in shaping the performance and availability characteristics of a fabric and the SAN. A
failure on an ISL can render the entire fabric useless, or an overloaded link can cause I/O
bottleneck and performance issues.
Therefore, it is necessary to have enough ISLs in the SAN to ensure adequate Availability and
Accessibility. Availability provides redundant paths to carry switch traffic. Accessibility
provides the bandwidth resources needed to support the host application.
Performance on a storage port is highly dependent on the number of I/O requests per
second and the size of each I/O request. As a general rule of thumb, avoid the use of ISLs, if
possible, or if it is absolutely necessary to use them, make sure there are enough of them.
For example, Backups are usually considered I/O intensive. When data is sent from a host or
storage to a tape drive, it can impact performance significantly. When this data has to
traverse an ISL, it is usually recommended that two ISLs be used. Traffic generated by four to
six tape drives can possibly flood an ISL with streaming data.
Note: This can vary based on the tape format used, the type of backup software, and the
CPU/memory capabilities of the host.
E_D_TOV It is the amount of time that the switch waits for an expected response before
declaring an error condition.
Set the same value for E_D_TOV on all switches and directors in a multiswitch fabric. If the
value is not the same, the fabric segments.
R_A_TOV It is the amount of time given to devices to allocate the resources needed to
process received frames. This variable works with the E_D_TOV variable to control the switch
behaviour when an error condition occurs. Resources are allocated to a circuit when errors
are detected and are not released for reuse until the time set by the R_A_TOV value expires.
Set the same value for R_A_TOV on all directors and switches in a multiswitch fabric. If the
value is not the same on all units, the fabric segments. Also, the value for R_A_TOV must be
greater than the value configured for E_D_TOV.
BB_credits - Buffer-to-buffer credits are a flow control mechanism to ensure that FC
switches do not run out of buffers, because switches must not drop frames. BB_credits are
negotiated on a per-hop basis.
Domain RSCNs - Domain registers for state change notifications (domain RSCNs) are sent
between end devices in a fabric to provide additional connection information to host bus
adapters (HBA) and storage devices. As an example, this information might be that a logical
path has been broken because of a physical event, such as a fiber optic cable being
disconnected from a port. Consult with the appropriate HBA and storage device vendor to
determine if enabling Domain RSCNs will cause problems with HBA or storage products in
the environment. For example, some host bus adapters (HBAs) may log out, then log back
into the fabric when they receive an RSCN, thereby disrupting Fibre Channel traffic.
Suppress RSCNs on zone set activations - Fabric format domain RSCNs are sent to ports on
the switch following any change to the fabrics active zone set. These changes include
activating and deactivating the zone set, or enabling and disabling the default zone. When
the Suppress RSCNs on Zone Set Activations check box contains a check, fabric format RSCNs
are not sent for zone changes to the attached devices on the Director. Enabled by default.
Director Speed - This option sets the system-wide internal data speed through the Director.
This speed displays in the Director Speed field of the Director Properties dialog box. If the
Director model does not support two data speeds, this option does not appear on the dialog
box.
One switch in the fabric is responsible for the distribution of Domain IDs, and plays a role in the route
creation for fabric management traffic. This switch is known as the principal switch. Since this switch
must communicate with all other switches and is the basis for fabric traffic routing, this switch should
be centrally located. This assists in the uniform delivery of information in the fabric, and provides a
consistent response to fabric build events.
Principal switches are selected during both the creation of the fabric and during fabric reconstruction
events. Two pieces of information located on the switch determine the selection of the principal
switch in the fabric. The Domain ID negotiation is governed by the entity in the fabric called the
principal switch. Principal switch selection is made based on the switch with combination of the
highest principal switch priority and then the lowest World Wide Name.
Proper placement of the principal switch in the fabric can lead to short negotiation times, resulting in
the fabric returning to a normal state in an shorter amount of time. For these reasons, EMC
recommends that the switch at the logical center of the fabric be made the principal switch. This
would normally mean the switch with both the least amount of hops to the farthest extent of the
fabric and/or the switch that has connections to the most other switches in the fabric. These two
placement strategies help to ensure that your principal switch access times are as quick as they can
be. The basic rules of Principle Switch selection are:
10
Switches in the fabric communicate through EFP (exchange fabric parameter) frames to
determine the principal switch as follows:
If one (and only one) director/switch is set to Principal, it becomes the principal
switch. This switch remains the principal switch as long as it is on line in the fabric,
even if someone changes another switchs Switch Priority to Principal.
If more than one director/switch have the same priority, the director/switch with the
lowest WWN number among those set to Principal becomes the Principal switch.
If no director/switch is set to Principal but one or more are set to Default, the
director/switch with the lowest WWN number among those set to Default becomes
the principal switch.
11
Each switch in a multiswitch fabric is identified by a unique Domain ID, in the range 1 through
239. Domain IDs are used in 24-bit Fibre Channel addresses to identify source and
destination ports in a fabric.
A switch can be assigned a domain-id-type of either preferred or static. The default domainid-type is preferred for all Fibre-Channel switches. In preferred mode, the domain-id is
dynamic and can be changed by the principal switch in the event of a domain-id conflict. A
static domain-id-type cannot be changed by the principal switch in the event of a domain-id
conflict. MDS-Series switches use the term static and B-Series switches use the term
Insistent for a static domain-id.
Some HBAs can handle a Domain ID change without requiring device driver administration
changes. In some operating environments, however, drivers require device driver-level
administrative changes when the Domain ID is changed.
If two operational fabrics join, they determine if any Domain ID conflicts exist between the
fabrics. If a conflict exists, the interswitch link (ISL) E_Ports segment themselves from each
other, preventing the fabrics from joining. To avoid this problem, assign a unique Domain ID
to each domain in a data center. If there are plans to go data center to data center, apply this
uniqueness across both data centers. This facilitates future fabric mergers.
12
13
The behavior for a subordinate switch changes based on the allowed domain ID lists, on the
configured domain ID, and on the domain ID that the principal switch has assigned to the
requesting switch:
When the received domain ID is not within the allowed list, the requested domain ID
becomes the runtime domain ID and the VSAN or VF is isolated on that Inter-Switch Link
(ISL).
When the assigned and requested domain IDs are the same, the options are not relevant,
and the assigned domain ID becomes the runtime domain ID.
When the assigned and requested domain IDs are different, and the configured type is
static or insistent, the assigned domain ID is discarded, all local interfaces are isolated, and
the local switch assigns itself the configured domain ID, which becomes the runtime
domain ID.
14
The fabric-capable device drivers on host bus adapters (HBAs) discover devices by using a
Name Server, which is a fabric-based function accessible to all nodes on the fabric via a wellknown fabric address.
When an N_Port logs onto the fabric, it registers its World Wide Port Name and FC-4 type
(SCSI) with the Name Server. A driver performing device discovery queries the Name Server
for a list of all SCSI devices that have registered with it. The driver uses this list to attempt a
login to each of the registered devices. In this manner, a fabric device driver discovers all
logged-in storage ports attached to the switch.
15
Oversubscription Ratio, as it applies to an ISL, is defined as the number of nodes or ports that
can contend for its bandwidth. This is calculated as the ratio of the number of free, non-ISL
ports to the number of ISLs on a switch. Oversubscription ratio is a theoretical number and
does not in anyway indicate the actual utilization of an ISL.
Methods used to aggregate ISLs are vendor proprietary and are not defined by the Fibre
Channel specification. Brocade, for example, calls this feature Trunking.
16
Connectrix Directors and switches use a number of algorithms to distribute the load over a
set of ISLs. These algorithms include:
A hop is a direct path between two switches with no other switches in between. A link
between an end device and a switch does not count as a hop.
The FSPF algorithm creates routes of minimum cost throughout the fabric. Every entry port
into a switch has a designated exit for each of the possible destinations. These exit
assignments change only if there is a reconfiguration event in the fabric. This single-static exit
ensures in-order delivery of the frames within an exchange. If an ISL or a new switch is added
or removed, the assignments may change. These assignments are done with logged-in
N_Ports. No routing assignments are done for F_Ports that have no logged-in ports.
When there are multiple equal-cost paths between a source and a destination, the routing
algorithm spreads the load over the multiple paths in a round robin fashion, sequentially
assigning a set of entry ports to a set of exit ports so that each exit port has the same
number of assigned entry ports (or a difference of only one).
17
Frames are routed across the fabric using an algorithm called Fibre-Shortest-Path-First (FSPF)
routing. Each ISL in the fabric is assigned a cost based on the link speed it is currently
operating at. FSPF calculates the shortest distance between an ingress and egress port by
finding the path with the lowest cost. The path with the lowest cost will typically be the
one(s) with the least number of hops, but this may not always be the case when mixing
different line rates (4 Gbps and 1 Gbps especially) in the same fabric. FSPF provides path
failover. Switch vendors also provide an ISL Trunking (also known as Port Channel in MDSSeries switches) feature that enables them to perform load balancing. Another benefit of ISL
trunking is a trunk appears to FSPF as a single entry on its database. This reduces the
database size and the frequency of link updates in it.
18
By default, only the active zoneset is distributed when the zone configuration is changed, or
when two fabrics merge. Therefore, the full zone database is resident on a single switch only.
19
This lesson covered an overview of fabric configuration. Build process, fabric entities and
path selection are covered.
20
This lesson covers B-Series fabric merge and expansion requirements and ISL configuration.
21
In B-series multiple VFs can be merged between different switches. Same-numbered logical
switches can merge with each other through the use of XISLs. In our example above we have
the physical switch, this switch doesnt have any configuration as everything happens in the
logical switches.
Logical switch with VF 5 will merge in pod 1 and pod 2 as long as the zone sets are
compatible and domain IDs are different. To allow the switch merger enable XISL use for the
logical switch.
Logical switch with VF 10 must be promoted to become the base switch. This switch can only
contain ISL connectivity. This switch wont have an active Zoneset. Timer configuration and
buffer-to-buffer credits will be configured here, as well.
22
To configure fabrics in B-Series switches with virtual fabrics, move to the virtual fabric switch
that has been promoted to base switch. This switch contains only ISLs. Disable the base
switch to receive all configuration options.
There are a number of settings in B-Series switches that control the overall behavior and
operation of the fabric. Some of these values, such as the domain ID, are assigned
automatically by the fabric and can differ from one switch to another in the fabric By
default, domain ID in B-series switches is 1. Other parameters, such as the BB credit, can be
changed for specific applications or operating environments, but must be the same among
all switches to allow the formation of a fabric. Use the configure command to change
those settings. To access all parameters controlled by this command, the switch must be
disable, if executed on an enabled switch, only a subset of attributes are configurable.
The following fabric parameters must be identical on each switch for a fabric to merge:
Data field size
Sequence level switching
Disable device probing
Suppress class F traffic
Per-frame route priority
Long distance fabric
R_A_TOV
E_D_TOV
23
In B-series switches domain IDs can be assigned dynamically when a switch is enabled,
however, it is possible to change them manually so that you can control the ID number or
resolve a domain ID conflict when you merge fabrics. The default domain ID for B-Series
switches is 1.
The switch must be disabled to set the domain ID . Issue the configure command to set the
unique domain ID. The value must be 1 through 239 for normal operating mode. Once the
domain ID is set, the switch can be enabled.
24
25
EMC recommends that you always attempt to give each switch a unique preferred Domain
ID, whether it is targeted for an existing fabric or a new fabric. (The exception to this rule
would be if you were swapping a new switch for a failed switch.) This provides the best
opportunity to merge a switch or fabric into another fabric with the least amount of manual
intervention.
Use the fabricshow command to display information about switches in the fabric. If the
switch is initializing or disabled, the message no fabric is displayed.
If the fabric is reconfiguring, some or all switches might not be displayed; otherwise, the
following fields are displayed:
26
You can compare zone databases against one another to identify any and all differences
between their membership prior to sending them to the switch or performing a merge. Once
the two databases have been compared, icons display to show the differences between the
two databases.
Select Configure > Zoning.
27
28
This lesson covered B-Series fabric merge and expansion requirements and ISL configuration.
29
This lesson covers MDS-Series fabric merge and expansion requirements and ISL
configuration.
30
In MDS-Series multiple VSANs can be merged between the same switches. Only samenumbered VSANs can merge each other. In our example above we are using VSAN 5 and
VSAN 10 in two different switches. The physical switches contain timer configuration such as
E_D_TOV and R_A_TOV. In MDS-Series this configuration are global to the switch and can not
be configured per VSAN.
VSAN 5 contains host and storage configuration for the two pods. The challenge with VSAN 5
is a Zoneset is configured. Zoneset incompatibility can lead to the segmentation of fabrics.
Different domain ids are assigned to each VSAN in order for them to merge. Notice VSAN 5
doesnt contain any ISLs. This is because the ISL trunking feature in MDS-Series switches will
be configured in VSAN 10. ISL trunking allows multiple VSANs to charge the same physical
ISL. VSAN 10 will only contain ISL connectivity. It is best practice to set trunk mode to on in
both switches. However, the configuration will work if one is set to on and the other to
auto. The trunk-allowed list contains all the VSANs that will share the ISL. VSAN 5 and 10
are added to the list so this ISL merges both VSANs between the switches.
Port configuration also includes buffer to buffer credits, they should be the same between
port pairs. The port mode is configured as E. Since some MDS-Series switches contain oversubscription, the ISL port rate mode must be dedicated.
31
In MDS-Series switches, domain IDs uniquely identify a switch in a VSAN. A switch may have
different domain IDs in different VSANs. Other parameters such as BB_credits and fctimers
can be changed to ensure a successful merge. The fctimer command is used to change
the fctimer values in the switch.
The E_D_TOV value has a valid range between 1,000 to 10,000 milliseconds. The
default is 2,000 milliseconds.
The R_A_TOV value has a valid range between 5,000 to 10,000 milliseconds. The
default is 10,000 milliseconds.
Those values are matched with the other end during port initialization.
The receive BB_credit (fcrxbbcredit) value may be configured for each FC interface and
depends on the module type and the port mode.
32
The port(s) used as ISLs need to be configured identical on both switches. The trunk mode
for the ISL should be set to on. This allows the ISL to be shared across different VSANs. In our
example we want to merge VSAN 10 containing our ISLs and VSAN 5, containing our hosts
and storage ports. When we use the switchport trunk allowed vsan 10
everything in the trunk allowed list is cleared and the only VSAN that can use the ISL is VSAN
10. To add VSANs to the list use switchport trunk allowed add vsan 10.
Notice, the word add is added to the command. If the word add was not used with the
command, the allowed list will be deleted and the only VSAN that could use the ISL would be
VSAN 10.
MDS-Series switches require ISL rate-mode to be dedicated. For other types of ports the
bandwidth can be shared across port-groups. Once the rate-mode is set to dedicated the
port can be created as an E_Port.
33
In MDS-Series, the default switch priority is 128. To set a different value, use the fcdomain
priority command. The valid range to set the priority is between 1 and 254 when 1 is the
highest priority. The switch priority value of 255 is accepted from other switches, but can not
be locally configured
This command sets the current VSAN 1 priority to one and attempts to cause a fabric rebuild.
34
The static example above, configures the switch in VSAN 10 to accept only a specific value
and moves the local interfaces in VSAN 10 to an isolated state if the requested domain ID is
not granted.
The preferred example, configures the switch in VSAN 10 to request a preferred domain ID
20 and accepts any value assigned by the principal switch. The domain is range is 1 to 239.
You must issue the fcdomain restart disruptive vsan <vsan> command to
apply the configured domain changes to the runtime environment.
Like most CLI commands, domain ID commands can be negated by preceding the command
with the no string, no fcdomain vsan <vsan>.
35
To verify that each switch is able to see the other switches, use the command show
fcdomain domain-list vsan <vsan>. If the command does not include a specific VSAN number,
the display lists the output for all VSANs.
The output of the command lists the set of domain IDs and associated WWNs for each
switch within a VSAN. This list provides the WWN of the switches owning each domain ID
and the information about whether a switch is the principal switch of the switches in the
fabric or VSAN to which it belongs.
The example above shows two switches in VSAN 1. This indicates that the switch where the
command was issued has built its adjacency in VSAN 1, with the other switch in the same
VSAN.
36
Domain information can be viewed and configured from the Device Manager > FC > Domain
Manager menu. Use the Running tab to view runtime values for the selected switch and the
Domains tab to view domain information for all switches in the VSAN. The Configuration tab
can be used to change configured values and initiate disruptive or non-disruptive restarts of
the fcdomain process.
37
Before joining two fabrics together, check that the Zone Sets in the two fabrics are
compatible. Remember that incompatible Zone Sets cause the two fabrics to remain
isolated.
Once youve merged two MDS-Series switches into a single fabric, in DCNM-SAN you now
see the two switches linked with an ISL. Click Zone, then Merge Analysis to see details of
how the merge happened.
38
As each VSAN ID is entered, click Analyze. The tool checks to see if the Zone Sets are
compatible and reports whether the merge will succeed or fail.
In the example above, there are two switches that are joined into a single physical topology.
The Zone Merge Analysis tool reports that VSANs 15 and 25 will have successful merges, but
VSAN 101 will not merge zonesets as there is no active zoneset on that VSAN.
39
40
This lesson covered MDS-Series fabric merge and expansion requirements and ISL
configuration.
41
This lesson covers native iSCSI configuration in the host and storage array.
42
To ensure data reaches all the users who need it, organizations are now looking for ways to
transport data throughout the enterprise locally over the SAN as well as over much longer
distances. One of the best ways to achieve this goal is to interconnect geographically
dispersed SANs through reliable, high-speed links. This approach involves transporting Fibre
Channel block data over existing IP infrastructures currently used throughout the enterprise.
The FCIP protocol standard has rapidly gained acceptance as a manageable, cost-effective
way to blend the best of both worlds: Fibre Channel block data storage and proven, widely
deployed IP infrastructure. As a result, organizations now have an excellent way to protect,
store, and move their data while leveraging existing technology investments.
43
The FCIP tunnel (or link) consist of one or more independent connections between two FCIP
ports. Each tunnel transport encapsulated Fibre Channel frames over TCP/IP. Although some
fabric vendors support multiple tunnels from a single GbE interface, it is recommended to
build parallel tunnels on different ports, modules, and switches to raise the high availability
coverage These equal-cost paths can be used by the Fibre Channel FSPF routing protocol for
load-balancing and recovery purposes.
44
FCIP entities are switches or other network adapters used during FCIP. The primary purpose
of an FCIP entity is to forward FC frames. Primitive signals, sequences and Class 1 FC frames
are not transmitted through FCIP because they can't encode using FC Frame encapsulation.
An IP network sees the FCIP entities as peers, therefore requiring TCP/IP communication.
FCIP entities contain 1+ TCP endpoints in IP-based network. From a Fibre Channel
perspective, the pairs of FCIP entities and their FC entities, forward FC frames between FC
fabric elements. The end nodes dont know an IP link exists, therefore the path taken by the
FC frames follow the normal routing procedure established by IP network. FCIP doesnt
participate in the FC frame routing.
45
FCIP can transport existing Fibre Channel services across the IP network such that two or
more interconnected SANs can appear as a single large SAN and can be managed by
traditional SAN management applications. In addition, FCIP enables SAN applications to
support additional protocols without modification. These applications might include disk
mirroring between buildings in a campus network or remote replication over the WAN. The
type of applications utilized are based on the distance the data must travel, the network
bandwidth, and the QoS requirements and/or abilities of the network connection.
While some implementations of FCIP are point-to-point tunnels, the protocol does not
require that the gateways support only point-to-point tunnelling. The FCIP standard
supports all Fibre Channel services, including FSPF routing algorithms, such that multiple
logical links created from a single gateway can route Fibre Channel packets over the IP
infrastructure. Not only is FCIP routable, but IP networks do not need to know anything
about the packets being routed.
The Fibre Channel services handle all routing between logical links, while the TCP protocol
handles the delivery of packets to the specific gateway device.
46
47
The MDS 9222i Multiservice Modular Switch delivers state-of-the-art multiprotocol and
distributed multiservice convergence, offering high-performance storage area network (SAN)
extension and disaster recovery solutions, intelligent fabric services such as Storage Media
Encryption, and cost-effective multiprotocol connectivity. With a compact form factor, the
modularity of the expansion slot and advanced capabilities normally available only on
Director-class switches, the MDS 9222i is an ideal solution for departmental and remote
branch-office SANs requiring the features present in a Director but at a lower cost of entry.
The MDS Multiprotocol Services Module delivers the intelligence and advanced features
required to make multilayer storage area networks a reality. Supported in the MDS Series
and offering fourteen Fibre Channel ports and two Gigabit Ethernet ports, the Multiprotocol
Services Module enables FCIP for long distance SAN extension and iSCSI for Ethernet
attached servers without sacrificing Fibre Channel port density. With its multiprotocol
capability, the module also integrates FICON protocol, FICON Control Unit Port (CUP)
management and switch cascading to enable mainframe connectivity.
The module is hot-swappable and includes hot-swappable, Small Form-Factor Pluggable
(SFP), LC interfaces for both Fibre Channel and Gigabit Ethernet. Individual ports can be
configured with short-wave, long-wave, or extended-reach SFPs for connectivity up to 100
kilometers. The Gigabit Ethernet interfaces are configurable for both FCIP and iSCSI
operation.
The SSN supports SAN extension with FCIP and Storage Media Encryption (SME). In addition,
the SSN delivers SAN extension performance with FCIP acceleration features including FCIP
write acceleration and FCIP tape write and read acceleration. The SSN supports hardwarebased encryption (with IP Security (IPsec)) and also supports hardware-based compression.
48
Each tunnel transport encapsulated Fibre Channel frames over TCP/IP. FCIP defines virtual E
(VE) ports, up to three on each physical port (EMC only supports one tunnel per port), which
behave exactly like standard Fibre Channel E_Ports, except that the transport in this case is
FCIP instead of Fibre Channel.
VE_Ports connects to VE_Ports only. A virtual ISL is established over a FCIP tunnel and
transports Fibre Channel traffic. Each associated virtual ISL looks like a Fibre Channel ISL with
either an E_Port or a TE_Port at each end.
By default, the FCIP feature on any MDS-Series switch creates two TCP connections for each
FCIP tunnel, one for data frames and the second for Fibre Channel control frames, i.e. switchto-switch protocol frames (all Class F) frames. To enable FCIP on the module, a FCIP profile
and FCIP interface (interface FCIP) must be configured.
49
As part of tunnel configuration, its necessary to configure the FCIP Profiles and FCIP
Interfaces on each physical interface. The FCIP profile contains underlying information about
local IP address and TCP parameters. The profile defines the local connection points (IP
address and TCP port number) and the behavior of the TCP connections for all FCIP tunnels
that use this profile. The FCIP profile's local IP address defines the physical interface (Gigabit
Ethernet port) where the FCIP tunnels terminates. The FCIP interface defines the local end
point of the FCIP tunnel and a VE_Port interface. All the FCIP and E_Port parameters are
configured in context to the FCIP interface. The FCIP parameters consist of the TCP
configuration, peer information, number of TCP connections for the FCIP tunnel, and E_Port
parameters (trunking mode and trunk allowed VSAN list).
50
The VSAN technology involves the hardware-based addition of a frame tag enabling traffic to
be identified and isolated to its particular VSAN. VSAN-tagged frames can also be carried
across an FCIP environment using the EMC MDS-Series IPS Module.
By using the VSAN capability over FCIP, separate virtual fabrics, each with their own Fibre
Channel fabric configuration including routing, zoning, name services, and fabric
management can be selectively extended across the wide area. Each FCIP tunnel created
using the IPS Module can be configured to be a virtual Trunking E_Port (TE_Port) thereby
allowing the VSAN tagged traffic to traverse the FCIP tunnel.
51
FCIP configuration parameters need to be the same on both sides of the tunnel. Before
proceeding to configure, enable the FCIP feature on the switch. The virtual ports on the
switch should contain the IP information of their peer switch. There is no discovery on FCIP.
52
FCIP license is included in many flavors under SAN Extension over IP. Notice the MDS-9222i
includes the SAN Extension license therefore there is no need to have an additional license
installed.
53
The FCIP configuration should be performed on both switches. Enable the FCIP feature on
the MDS-Series switch. Next, assign an IP address in the physical gigE port on the switch and
enable the port.
Configure an FCIP profile and bind it with the physical gigE port by assigning the IP address of
the physical gigE port. Within the profile configuration sub-mode, create an FCIP interface,
this is the equivalent of a VE_Port. Bind the FCIP interface with the FCIP profile created
above and configure the IP address of the peer switch. Enable the FCIP interface. Once the
configuration has been created on the other switch the ISL will be formed.
54
The FCIP virtual port behaves exactly the same as a regular ISL.
The port(s) used as ISLs need to be configured identical on both switches. The trunk mode
for the ISL should be set to on. This allows the ISL to be shared across different VSANs. In our
example we want to merge VSAN 10 containing our ISLs and VSAN 5, containing our hosts
and storage ports. When we use the switchport trunk allowed vsan 10 everything
in the trunk allowed list is cleared and the only VSAN that can use the ISL is VSAN 10. To add VSANs
to the list use switchport trunk allowed vsan add 10. Notice, the word add is
added to the command. If the word add was not used with the command, the allowed list is
deleted and the only VSAN that could use the ISL would be VSAN 10.
55
Licenses can be checked throughout different sections of DCNM-SAN and Device Manager. To
check for installed licenses in DCNM-SAN, from the Physical Attributes pane expand
Switches, and click Licenses.
56
To enable the FCIP license, double-click your switch from the topology pane to open Device
Manager. In Device Manager navigate to Admin > Feature Control.
From the Feature Control window, select the Control tab and under action set FCIP to
enabled. Apply the changes to save the configuration.
57
The DCNM-SAN FCIP wizard makes it very simple to configure FCIP between two switches. In
order to use the wizard DCNM-SAN Server must be installed since both independent fabrics
need to be discovered by DCNM-SAN. In our example an ISL is already connecting our fabrics
together, therefore DCNM-SAN Server is not required.
To start the wizard, click the icon on the toolbar. The first step in the wizard is to select the
switch pair. Select the two switches that will be used for FCIP.
58
For each switch in the switch pair, select the gigE ports that will be connected together in
step 2. Additional options can be configured such as IPSec security or using large MTUs for
better performance. Leave this option to the default.
In Step 3, we assign IP addresses for both switch gigE ports. Where necessary manually add a
GigE route for the IP address. The wizard automatically configures the peer IP address.
59
In Step 4, we can set bandwidth allocations. There can be multiple FCIP or iSCSI virtual ports
assigned to each physical gigE port. Therefore, the switch allows bandwidth allocation for
each virtual port. The maximum bandwidth should always be set to 1000 while the minimum
bandwidth should be a value thats always available for this virtual port. Having multiple FCIP
virtual ports or a mix of iSCSI and FCIP ports in one gigE port is not supported by EMC.
The last step automatically assigns a profile ID and FCIP port to each gigE port. Notice this
value can be changed by clicking the . VSAN trunking can be configured here as well. Add
the range of VSANs that can be trunked across the ISL.
Click Finish to complete the FCIP configuration.
60
This lab covers the merger of two MDS-Series fabrics over an IP network by using FCIP
protocol
61
This lesson covered MDS-Series fabric merge and expansion requirements and ISL
configuration.
62
This lesson covers native iSCSI configuration in the host and storage array.
63
The Internet Small Computer Systems Interface (iSCSI) protocol provides a means of
transporting SCSI packets over TCP/IP. iSCSI works by wrapping SCSI commands into TCP, and
transporting them over an IP network. Since iSCSI is IP-based traffic, it can be routed or
switched on standard Ethernet equipment.
64
The iSCSI feature consists of routing iSCSI requests and responses between iSCSI hosts in an
IP network and Fibre Channel storage devices in the Fibre Channel SAN that are accessible
from any Fibre Channel interface. Each iSCSI host that requires access to storage through the
switch needs to have a compatible iSCSI driver installed, such as the Microsoft iSCSI Initiator.
Using the iSCSI protocol, the iSCSI driver allows an iSCSI host to transport SCSI requests and
responses over an IP network. From the host operating system perspective, the iSCSI driver
appears to be a SCSI transport driver similar to a Fibre Channel driver in the host.
The iSCSI capable switch provides transparent SCSI routing. IP hosts using the iSCSI protocol
can transparently access targets on the Fibre Channel network.
The switch creates a separate iSCSI SAN view and Fibre Channel SAN view. For the iSCSI SAN
view, the switch creates iSCSI virtual targets and then maps them to physical Fibre Channel
targets available in the Fibre Channel SAN. They present the Fibre Channel targets to IP hosts
as if the physical iSCSI targets were attached to the IP network.
For the Fibre Channel SAN view, the switch presents iSCSI hosts as a virtual Fibre Channel
host. The storage devices communicate with the virtual Fibre Channel host similar to
communications performed with real Fibre Channel hosts.
The switch transparently map the command between the iSCSI virtual target and the virtual
Fibre Channel host.
65
Native iSCSI consists entirely of components that transmit the SCSI protocol over TCP/IP. In
this case, iSCSI initiators (host server) may be directly attached to iSCSI targets (Storage
array), or may be connected using standard Ethernet routers and switches.
Before implement an iSCSI native solution, you must consider:
Set up the IP address, configuring the iSCSI drivers and configure the targets on the
iSCSI initiator.
66
This driver is found in the Linux kernel. The iSCSI driver provides a host with the ability to
access storage through an IP network. The driver uses the iSCSI protocol to transport SCSI
requests and responses over an IP network between the host and an iSCSI target device. The
iSCSI protocol is an IETF-defined protocol for IP storage.
Architecturally, the iSCSI driver combines with the host's TCP/IP stack, network drivers, and
network interface card (NIC) to provide the same functions as a SCSI or a Fibre Channel (FC)
adapter driver with a host bus adapter (HBA). The iSCSI driver provides a transport for SCSI
requests and responses to storage devices via an IP network instead of using a directly
attached SCSI bus channel or an FC connection. The storage router, in turn, transports these
SCSI requests and responses received via the IP network between it and the storage devices
attached to it.
Two types of drivers are used for iSCSI in Linux environments. The iscsi-sfnet driver is found
in RHEL 3 and 4, Asianux 1.0, Asianux 2.0, SLES 8, and SLES 9.
The newer open-iscsi driver is found in RHEL 5 and 6, SLES 10, and SLES 11. The open-iscsi
driver is a high-performance, transport independent, multi-platform implementation of
RFC3720 iSCSI. Open-iscsi is partitioned into user and kernel parts. The kernel part
implements iSCSI data path (that is, iSCSI Read and iSCSI Write). User space contains
configuration manager, iSCSI discovery, login and logout processing connection-level error
processing.
The user space open-iscsi consists of a daemon process called iscsid and a management
utility iscsiadm.
67
To manually stop the iSCSI driver for RHEL and Asianux, enter:
# /etc/init.d/iscsid stop
You must make sure that anything that has an iSCSI device open has closed the iSCSI device
before shutting down iSCSI. This includes file systems, volume managers, and user
applications.
If iSCSI devices are open when you attempt to stop the driver, the script errors out instead of
removing those devices. This prevents you from corrupting the data on iSCSI devices. In this
case, iscsid will no longer be running, so if you want to continue using the iSCSI devices, it is
recommended that you issue /etc/init.d/iscsi start.
68
Edit the /etc/iscsi/iscsid.conf file. There are several variables within the file. The
default file from the initial installation is configured to operate with the default
settings. The syntax of the file utilizes a pound (#) symbol to comment out a line in
the configuration file. You can enable a variable listed below by deleting the pound
(#) symbol preceding the variable in the iscsid.conf file.
69
To configure iSCSI on Linux set the run levels for the iSCSI daemon to automatically start at
boot and to shut down when the server is brought down.
# chkconfig -level 345 iscsid on
For RHEL or Asianux 3.0, you will need to perform a series of iscsiadm commands to
configure the targets you wish to connect to with open-iscisi.Consult the manpages for
iscsiadm for a detailed explanation of the command and its syntax.
1. Discover the targets you want to connect your server to via iSCSI. For VNX arrays you
only need to perform a discovery on a single IP address and the array returns all its
iSCSI-configured targets. However, for the Symmetrix array you need to perform the
discovery process on each individual target.
Before you perform the discovery of each port on the Symmetrix, you need to
configure the targeted Symmetrix iSCSI ports to accept the IQN of your initiator.
.# iscsiadm -m discovery -t st -p 51.51.51.203
70
VMware contains a software iSCSI initiator. In the same Storage Adapters section where the
HBAs can be found, click Add. When the window-pops up select Add Software iSCSI Adapter
and press OK.
71
When the iSCSI software initiator is added, you see the new vmhba. When you click it, the
WWN can be found.
72
When you click Properties, for your iSCSI configuration the IQN can be found. Notice the
initiator is enabled. From this windows you can configure CHAP security, advanced features
and use Configure to change the IQN, set an iSCSI Alias and/or enable the feature.
73
In order for iSCSI to work in ESXi you need to assign a physical adapter to use with iSCSI.
From the Network Configuration click Add and select the desired network card, then click
Ok. The IP configuration of the network card will be used to connect to the iSCSI server.
74
Now that the network has been configured use Dynamic Discovery to point to the storage
array. From the Dynamic Discovery tab, click Add and set the IP address of the storage array
port. Click Ok to register with the iSCSI ports in the storage array.
75
When the initiator sees the target, make the connection by clicking Log On.
Before start, verify the iSCSI initiator is installed. Versions of Windows such as Windows 2008
and Vista include the iSCSI initiator. Older versions require the driver be downloaded from
the Microsoft website and installed. To check if your version contains the driver simply open
Control Panel and look for the iSCSI Initiator icon. If it is not present, download and install it
from the Microsoft website. Sometimes the iSCSI Initiator service might not be running. If
you open the iSCSI initiator software and the service is not running, you are prompted to
start the service.
76
The first step is to configure the targets on the iSCSI initiator. This step tells the initiator
where to look for LUNs. Simply click Add in the Target Portals area, pointing the IP address or
DNS name and Port number for the iSCSI Target portal that you want to logon. If Advanced is
clicked, the preferences to set up digest methods, CHAP authentication etc. are displayed.
Not all sites require advanced setup procedures. When you click OK, the initiator attempts to
discover iSCSI targets.
If discovery is successful, an available portal is displayed on the target portal screen.
When using bridged iSCSI, the target is the IP address of the GigE port on the FC gateway
device that the hosts is connected to. For native iSCSI, the target is the port on the storage
array.
Note: With an MDS switch, the target portal is the IP address of the GigE port on the MSM
module. The MSM module takes Fibre Channel storage targets and emulates iSCSI storage
targets.
77
Select the Targets tab to display a list of available targets that you can log on to. There should
now be an available target listed here. The iSCSI name of this target should be the one you
were expecting. Note that the Status for these targets is shown as Inactive prior to logon.
Inactive means you don't have a session yet. Discovery session discovers the target name,
then closes the session.
If the targets are not listed on the Targets tab, verify discovery and successful logon
checking the Target portal on Discovery tab.
If you successfully logged onto the Target portal but the target is still not listed, verify
that the target has LUNs assigned to this server. This step must be performed in the
gateway switch or storage array, depending on the type of implementation. This step
is covered in next slides.
78
Select the target that you want to log on to and click Log On. When logging on to a target,
make sure to check the Automatically restores this connection when the computer starts box
to make the connection persistent. This allows the host to reconnect to the iSCSI target after
a reboot. If we click Advanced here, we go to the same Advanced dialog as previously
displayed.
79
Select the target that you logged on to and click Details to display the Target Properties.
Note the Session Properties Status and Connection Count.
Select Devices to view the LUNs presented within this session. If you want to see device
details, select the device and click Advanced.
80
Right-click the storage system to display the properties. The Unique ID (IQN) of a iSCSI
storage system. iSCSI iqn is a predefined name for an iSCSI storage system, or the iSCSI name
for a combo storage system.
This can be useful in helping you to identify the correct CLARiiON Storage array that your
iSCSI ports are belong to as the array serial number is embedded in the iqn identifier.
81
Right-click the storage system and select Port Management. Select the iSCSI port and click
Properties. The IQN can be determined for the port, as well as other parameters such as
speed and initiator information. Here, you can set the Alias and IP Address.
82
Register iSCSI connections, configure LUNs and Storage Groups in the same manner as for
Fibre Channel array. More of this step is explained in the SAN Management module.
83
Configuring iSCSi ports can be done by selecting the Settings menu option then Network.
This launches the Ports page. All Port information can be viewed from this page. Locate and
click the iSCSI port under Types. The Properties tab is highlighted and the Port Properties
page appears.
Port information can also be viewed from the Hardware view by locating and right-clicking
the respective port under the I/O modules tree.
84
From the iSCSI Port Properties window click Add and supply the Virtual Port parameters.
The IP address of the selected virtual port uses a 32-bit numeric address written as four
numbers separated by periods, for example 123.221.34.33. Network names are NOT
supported.
The Gateway address uses a 32-bit numeric address written as four numbers separated by
periods, for example 123.221.36.1. Network names are NOT supported.
The Subnet Mask uses a 32-bit numeric address written as four numbers separated by
periods, for example 255.255.255.0.
Use caution when configuring the VLAN Configuration option. Only network administrators
should assign VLAN IDs. If you enter an incorrect value, you may lose contact with the
storage system.
When VLAN Tagging is enabled, users can assign a VLAN ID to the selected virtual port.
VLAN ID Lets you assign the VLAN ID to the virtual port (must be between 1 and 4094). VLAN
ID must be unique for that port.
The Require initiator authentication box enables iSCSI initiator authentication, users must
enable initiator authentication in order for CHAP security to work.
85
SMC gives the ability to view and set the IP addresses on GigE ports. The General tab on the
Properties of the port displays the iSCSI name, IP address, Gateway and Netmask.
To set the IP address, right-click the port in the Navigation Tree, select Port and Director
Configuration, and then Set Port Attributes. From here you can set the IP address, Gateway
and Netmask for your SE ports.
Configure Storage Provisioning in the same manner as for Fibre Channel array. More of this
step is explained in the SAN Management module.
86
With few exceptions, if the underlying Ethernet network is functioning properly, iSCSI
performs remarkably fast. Generally, it is recommended to segment off the iSCSI traffic so it's
not routed or mixed with public traffic but unless there is network saturation there shouldnt
be any issues.
To simplify the fabric and avoid network congestion, all switch and host ports in the SAN
should be configured for the highest-speed full-duplex operation, overriding any autonegotiation functionality. Full-duplex operation allows the switch and host to exchange data
bidirectional at the same moment in time, as compared to half-duplex operation which
requires that transmission occur in only one direction at a time. In half-duplex operation,
simultaneous transmission is termed a collision. The packets are discarded and must be
retransmitted. Half-duplex communication is required when the physical medium lacks
enough wires to accommodate bi-directional signaling, such as coaxial cable, or when nonintelligent network equipment is used. Neither of these conditions should exist in a modern
IP network designed to carry storage traffic.
87
LAN configuration allows Layer 2 (switched) and Layer 3 (routed) networks. Layer 2 networks
are recommended over Layer 3 networks.
The network should be dedicated solely to the iSCSI configuration. For performance reasons,
EMC recommends that no traffic apart from iSCSI traffic should be carried over it. If using
MDS switches, EMC recommends creating a dedicated VSAN for all iSCSI traffic.
CAT5 network cables are supported for distances up to 100 meters. If cabling is to exceed
100 meters, you must use CAT6 network cables.
The network must be a well-engineered network with no packet loss or packet duplication.
When planning the network, care must be taken in making certain that the utilized
throughput will never exceed the available bandwidth.
vLAN tagging protocol is not supported, nor is Link Aggregation, also known as NIC teaming.
88
By default, Path Maximum Transmission Unit (PMTU) discovery is disabled on MDS switches.
When PMTU discovery is disabled, an MTU of 1500 bytes is used for all remote destination IP
addresses. The PMTU can be adjusted to take advantage of jumbo frames support, namely
Ethernet frames that support a frame size greater than the IEEE standard of 1518 bytes.
Using jumbo frames, a payload can be configured to completely fit a full-size SCSI frame
thereby eliminating the requirement to fragment the frame into two TCP packets. In order to
use jumbo frames support, the entire IP infrastructure must support jumbo frames.
Jumbo frames extend Ethernet's bytes per frame size up to 9,000 bytes. The size is necessary
because Ethernet uses a 32-bit CRC that loses its effectiveness above about 12,000 bytes. It
is also necessary because 9,000 is large enough to carry an 8KB application datagram plus
packet header overhead. Changing the default frame size from an MTU size of 1,500 to a
higher value, i.e. 9000, can significantly increase performance on the IP SAN.
89
It is critical that the storage array be properly sized to meet the anticipated needs from all
active hosts. Array sizing requires using two different perspectives:
Do we have sufficient raw capacity of usable storage? When sizing the number of disks for
anticipated throughput rate, you may need to provision significantly more raw GB of data
than what is strictly required or specified.
Will the system meet the I/O throughput performance requirements of all existing and
newly-added hosts? The performance of an iSCSI target is usually characterized in terms of
throughput, expressed in megabytes per second and I/O operations per second. Like a disk
drive, the performance of an iSCSI target is greatly affected by the I/O access pattern.
Sequential I/O that can be streamed to or from the target's disks will run faster than random
I/O that causes those disks' heads to seek. Moreover, most targets provide an ample disk
block cache, and I/O that can be satisfied from that cache will be faster than I/O that must go
all the way to the target's disks. Unlike a disk drive, however, the performance of an iSCSI
target is also affected by the characteristics of the network path dealing with available
bandwidth and latency between the initiator and target.
90
PowerPath iSCSI is built on the MPIO framework. It provides NO load balancing and is
intended to support Microsoft Operating Systems.
PowerPath Classic also works with both FC and IP SAN:
91
All hosts with NICs require an EMC-qualified version of the iSCSI initiator software, which are
typically free downloads for supported components.
iSCSI initiator host software is also available for other operating systems (e.g. HP-UX), but
these are not supported.
iSCSI servers can attach via Network Interface Card (NIC) or iSCSI HBA.
Standard NIC solution requires more CPU utilization.
iSCSI HBAs provide TCP/IP and/or iSCSI offload at a cost.
Refer to the support matrix for the EMC-qualified firmware and driver versions.
92
This lab covers native iSCSI configuration between Windows and Linux hosts and VNX or
CLARiiON Arrays.
93
This lesson covered bridged and native iSCSI configuration in the host, switch, and storage
array.
94
This module covered how to configure multiple switches in a fabric. The module also covered
iSCSI configuration.
95
96
Interoperability is the term used to describe a Fibre Channel fabric that contains switches
from more than one vendor. Most vendors adhere to the ANSI T11 FC-SW-2 specification for
Fibre Channel switch interoperability, but not all. This results in switch interoperability issues,
which prevent customers from building heterogeneous fabrics. To address this issue, switch
vendors support a special mode, which, when enabled, allows them to interoperate with
other switches.
Each vendor should have a regular mode and an equivalent interoperability mode, which
specifically turns off advanced or proprietary features and provides the product with a more
amiable standards-compliant implementation. Each vendor, in this case MDS-Series, B-Series,
and M-Model, has its own normal mode and an equivalent interoperability mode, which has
the purpose of turning off specific advanced or proprietary features, and providing the
product with a more standards-compliant implementation.
B-series Fabric OS v6.0.0 and higher supports changing between interop modes using
Interopmode 0, 2 and 3. On Fabric OS v6.0.x and higher, it is no longer supported to
configure switches for interopmode 1, which supported non-B-Series switches, for example
MDS-series and QLogic. McDATA Open Fabric mode, interopmode 3, replaces interopmode 1.
McDATA Open Fabric mode is intended specifically for adding Fabric OS-based products to MEOS fabrics that are already using Open Fabric mode. Fabrics containing only Fabric OS
switches in Open Fabric mode are not supported.
M-EOS products must operate with the most recent version of M-EOS supported for
interoperability. M-EOS v9.7.2 is the minimum version of firmware that is fully qualified to
interoperate with Fabric OS v6.2.0 or later. For support of Frame Redirection in McDATA
Fabric Mode (interopmode 2), M-EOS products must use v9.8 or later. For support of Frame
Redirection in McDATA Open Fabric Mode (interopmode 3), M-EOS products must use v9.9
or later. Only the ES-4400, ES-4700, M6140, and Mi10k switches can have devices directly
attached that are having data encrypted or unencrypted.
Check the EMC Support Matrix for the latest interoperability support information.
The standard interoperability mode 1 feature allows switches to interoperate with each
other while configured in interop and open modes. Using standard or heterogeneous
interoperability requires enabling interoperability mode on all legacy switches.
Unfortunately, the process of enabling the interoperability mode on legacy switches results
in fabric-wide disruption and loss of existing functionality. For customers deploying SANs in
mission critical environments, this might not be an acceptable solution.
MDS-Series switches address this issue with a legacy switch interoperability mode feature
that can be set to interoperate with an installed base of specific legacy switches without
disrupting the existing fabric services or changing configuration on the legacy switches. This
functionality enables storage administrators to consolidate SAN islands while preserving their
existing investments in legacy switches.
The primary difference between interop mode 3 and mode 2 is how the B-Series switch sets
the core process ID (PID). When B-Series switches have a higher port count than 16, the core
PID is set to 1. B-Series initially, pre-16-port switch, allocated one nibble of the Fibre Channel
ID/PID, or FCID/PID, in area field 0x0 F for the port number, thus limiting port count to 16.
When the core PID is set to 1, the allocated bytes in the FCID or PID allow for the use of port
numbers 0x00 FF. Therefore, for VSANs running in interop mode 3, the core PID on the BSeries switch is set to 1.
The legacy switch interoperability mode 3 for Connectrix B switches with more than 16 ports
(and a core PID =1) was introduced with Connectrix MDS SAN-OS Release 1.3. With this
VSAN-based interop mode, Connectrix B switches do not have to be altered from their native
mode and can be seamlessly added to a new or existing Connectrix MDS SAN-OS VSAN.
Legacy switch interoperability mode 4, provides the means to non-disruptively connect a MModel switch running in McDATA Fabric 1.0 mode with an MDS-Series switch. While in this
mode, the VSAN emulates the behavior of a M-Model switch, including the use of offsets
when referring to domain IDs and FC IDs, and in its ability to only establish an Inter-Switch
Link (ISL) with another switch that uses McDatas OUI (08:00:88). EMC supports Interop-4
when using EOS 9.7.2 and SAN-OS v3.3.1c, and also when using EOS 9.8.1 and NX-OS 4.1.1b.
Here is a matrix with the different interop modes supported. The scenarios can change
depending on the switches and code used. For this reason, it is recommended to check the
EMC Support Matrix for the latest interoperability support information.
Some of the features that are available prior to enabling the interoperability mode on FC switches are
subsequently disabled. This creates operational challenges for storage administrators who have to
give up functionality in order to build heterogeneous fabrics. Some features that are not available on
the specific vendor switches when operating in Interop Fabric Mode include:
B series switches
QuickLoop
QuickLoop Fabric Assist
Remote Switch
Extended Fabrics
Trunking
Secure Fabric OS
Alias Server
Platform Service
Virtual Channels
FCIP
MDS switches
TE_Ports (trunking expansion ports) and Port-Channels cannot be used to connect MDS to
non-MDS switches. However, TE_Ports and Port Channels can still be used to connect an MDS
to other MDS switches even when in Cisco Fabric mode VSANs.
The Quality of Service feature is intended to provide nodes with high bandwidth needs and
greater access to the fabric resources. Quality of Service is applied end to end (host to
storage), and can be implemented only if host and storage are attached to MDS models.
M-Model switches
Show route (where targets or initiators are located on Brocade or Cisco switches)
Show zone (where members are located on Brocade or Cisco switches)
Fabric Binding
Enterprise Fabric Mode
SANtegrity (with the exception of Switch Binding)
Copyright 2013 EMC Corporation. All rights reserved
10
This checklist can be used to serve as a tool for completing all steps before actually merging
heterogeneous fabrics.
Verify that each switch has a unique Domain ID. When merging fabrics, ensure that
there are no duplicate Domain IDs among all switches that will be part of the merged
fabric.
Verify that all switches have been set up to work in a supported interop mode.
Verify that the E_D_TOV and R_A_TOV are set the same on all switches that will be
part of the new fabric. (By default, they should all be the same; if necessary, refer to
the appropriate user manual for information on how to set up operating parameters.)
Note: Switches use different units to represent the same values; for example a value
of 2000 on a Brocade switch or Cisco switch is the equivalent of 20 on a Brocade MModel switch.
Verify that the active zone set has been checked (with the respective switch fabric
management tools) and does not contain illegal characters.
If a switch is not operational and the zoning definition on that switch is not required,
be sure to clear the zoning configuration on that switch.
If a switch is operational and the zoning configuration on that switch is required, be
sure to check that there are no duplicate active zone names. If there are duplicate
zone names, rename one of the zones.
Ensure that all switches are configured with WWN zoning.
Ensure that all switches comply with proper zone naming.
Back up the switch configuration by issuing the appropriate commands.
11
The items on this checklist can be considered as recommended best practices to configure a
stable interoperable environment with minimal disruption to existing data flow, if any.
Set the Domain IDs rather than allowing the fabric to set them.
Set the core switch as a principal switch. This reduces Class F traffic by ensuring that it
goes directly from core to edge. For example, if an edge switch is the principal switch,
build fabric traffic must go through the core to get from edge to edge.
In a vendor switch migration from either B-series or M-model switches, leave them in
their native modes and use Ciscos legacy modes, Interop-3 and Interop-4 (only if
supported) for a non-disruptive and seamless migration.
12
The domain ID, which is part of the FCID, may be limited to a range less than the full 239
values provided in the Fibre Channel standard. A switch may have to change its domain ID to
the 97 to 127 range to accommodate the M-Model 31 domain address limitation. If a
domain ID is changed (which can be a disruptive event to the switch), all devices attached to
the switch will need to log into the switch again. When domain IDs are changed, the switch
itself will need to re-register with the principal switch in the fabric to verify domain ID
uniqueness.
DisruptiveThe impact of this event may be switch-wide. B-Series without Virtual Fabrics
and M-Models require the entire switch to be taken offline and/or rebooted when changing
domain IDs.
NondisruptiveThis event is limited to the VSAN where the event is taking place. MDSSeries switches can perform this action, as the domain manager process for this VSAN is
restarted and not the entire switch. This event still requires any devices logged into the VSAN
on that switch to log in again to obtain a new FCID.
13
14
15
In this scenario, M-Model switches are set to Open Fabric 1.0 mode (interop mode). ISLs are
created to link M-Model switches with B-Series switches, which will be operating in their
supported Interop mode 3. Before adding the B-series switch to the fabric, the following
steps need to be performed on the M-Model switch:
Verify switch firmware versions Verify that the firmware levels running on these
switches are supported in the EMC Support Matrix for the respective interop modes
Alter the timers, RA_TOV and ED_TOV, if needed. Notice, the timers are usually set
correctly and dont need to be altered.
16
The next slides are an example of how to configure an M-Model switch before it is
introduced in a heterogeneous environment.
Setting the interop mode on an M-Model switch is disruptive, so first disable the switch.
17
18
To set the interop mode go to Configure > Switch > Fabric Parameters.
19
For merged fabrics, zoning is managed by WWNs. Do not use fabric addressing,
domain, port number, or area zoning. If any method other than WWN zoning is used,
convert to WWN zoning.
Use of a M-Model director as the principal switch ensures fabric addressing is not
used. Also Enables fabric information to be obtained directly from the principal
switch (through the SAN management application).
20
This is a generic set of instructions that need to be configured on B-Series switches before
introducing them in an heterogeneous environment. Changing the Domain ID or setting
interop mode on B-Series switches, requires disabling the switch. With the creation of virtual
fabrics most configuration parameters take place exclusively on the logical switch without
affecting other logical switches in the fabric.
21
After verifying the fabric, create or configure a previously created logical switch. In order to
change the domain ID the logical switch must be disabled. Disable the logical switch and run
the configure command. Choose to configure fabric parameters and set the domain ID to
the desired value. When prompted to allow XISL use make sure to answer no since XISLs
are not supported in an interoperable fabric.
22
Make sure the following values are set in time-out values and buffer-to-buffer credits. When
the configuration is complete check the existing interop mode in the switch. In our example
interop mode is set to off. Set interop mode to 3 and enable the switch. The switch is now
ready to merge with an M-Model switch.
23
24
25
In this scenario, B-Series switches will be set to Interop mode 3 (Brocade's interop mode 1
has been replaced with interop mode 3 on Brocade FOS v6.0.x and higher), ISLs will be
created to link the B-Series switches with the MDS switches, which will be operating in their
supported Interop mode 1. Before adding the MDS switch to the fabric, the following steps
need to be executed on the B-Series switch:
Before merging the fabrics, ensure that all the B-series switches in the fabric have
supported firmware. Login to the switch as admin and verify the firmware version
using the Telnet version command.
Create or configure an existing logical switch, make sure XISL is disabled on the
switch.
Ensure that all the switches in both fabrics have unique domain Ids before the merge.
List the domain ID addresses of each switch in each fabric and verify there are no
duplicate Ids. If any duplicate IDs exist, change the IDs by assigning different domain
numbers.
Platform management services must be disabled fabric-wide before enabling the
interopmode and merging the fabrics. The msPlMgmtDeactivate command
deactivates the Platform Database Management Service of each switch in the fabric.
For B-series switches the interopmode 3 must be enabled before merging with MDSSeries switches. This command enables interopmode on individual logical switches
only and therefore, must be executed on each B-series logical switch in the fabric.
To ensure proper zoning merge and operation, verify there are no duplicate active
zone sets or zones across the two fabrics that are to be merged. If any duplicate
zones exist, rename them.
26
Before adding the MDS Series switch to the fabric, the following steps need to be executed:
Verify all MDS-series switches are configured with supported firmware as shown in
the EMC Compatibility Matrix.
Verify that the Fibre Channel timers are set to the system default values. The MDS
9000 and B-Series FC Error Detect (ED_TOV) and Resource Allocation (RA_TOV) timers
default to the same values.
To enable interoperability mode on MDS Series switches, you must place the VSAN of
the E_Ports that connect to the B-Series in interoperability mode.
To ensure proper zoning merge and operation, verify there are no duplicate active
zone sets or zones across the two fabrics that need to be merged. If any duplicate
zones exist, you must rename them. You can verify zoning information with the show
zone command.
27
The example above shows how to configure a MDS Series switch before to introduce it into
an heterogeneous switched fabric.
Here are some considerations when Interopmode 1 is enabled.
Set Interop Mode on desired VSAN.
All zone members are pWWNs using standard interop mode, because the B-Series/MSeries domain/port nomenclature is not a valid form (per the FC standard).
Trunking and PortChannel features are disabled. While in interop mode, the BSeries/M-Series switches do not support trunked ports of any type. Only standard E
ports are used for the ISLs.
The domain IDs are limited to 97-127, a restriction imposed by M-Models inability to
handle IDs outside of that range. A M-Series switch is not present in this
configuration, but the decision to have a single interoperability mode for the B-Series
and MDS Series switch causes this side effect. While B-Series switches and MDS
Series switches can handle domain IDs outside of this range, their implementation of
interoperability mode includes this limitation.
28
29
This lab creates a heterogeneous fabric by implementing interoperability between MDSSeries and B-Series switches
30
This module covered different interoperability modes available on different series switches. It
showed you how to configure interoperability between switches.
31
32
This module focuses on the virtualization features offered by the Connectrix switches,
including NPIV, NPV and Access Gateway.
Module 7: Virtualization
This lesson covers the NPIV feature and requirements to configure it in VMware
environments using Connectrix switches.
Module 7: Virtualization
N_Port ID Virtualization (NPIV) is a standard that describes how a single Fibre Channel HBA
port (single N_Port/single FCID) can register with several World Wide Port Names (WWPNs)
or multiple N_Port IDs in the SAN fabric. This allows a fabric-attached N_Port to claim
multiple fabric addresses. Each address appears as a unique entity on the Fibre Channel
fabric
In other words, NPIV-capable HBAs can provide multiple WWPNs rather than registering a
single WWPN in the fabric. This is beneficial in two ways: In a virtual machine environment
each VM can have separate WWPNs so that the hypervisor will be released to provide the
I/O blending operation. In a virtual machine environment where many host operating
systems or applications are running on a physical host, each virtual machine can now be
managed independently from zoning, aliasing, and security perspectives. Also, there would
be no extra physical ports to be connected in the SAN fabric so the addition of more edge
switches would not be required.
Module 7: Virtualization
When virtual machines have WWN assignments, they use them for all RDM traffic, so the
LUNs pointed to by any of the RDMs on the virtual machine must not be masked against its
WWNs. When virtual machines do not have WWN assignments, they access storage LUNs
with the WWNs of their hosts physical HBAs.
By using NPIV, however, a SAN administrator can monitor and route storage access on a per
virtual machine basis. NPIV enables a single FC HBA port to register several unique WWNs
with the fabric, each of which can be assigned to an individual virtual machine. When a
virtual machine has a WWN assigned to it, the virtual machines configuration file (.vmx) is
updated to include a WWN pair (WWPN and WWNN).
As that virtual machine is powered on, the VMkernel instantiates a virtual port (VPORT) on
the physical HBA which is used to access the LUN. The VPORT is a virtual HBA that appears to
the FC fabric as a physical HBA. Each VPORT is specific to the virtual machine, and the VPORT
is destroyed on the host when the virtual machine is powered off.
When a virtual machine using NPIV is powered on, it uses one of WWN pairs (Up to 16) in
sequence to try to discover an access path to the storage. In other words, each virtual
machine can have up to 16 virtual ports (or 16 WWN pairs, NPIV-aware virtual machines are
assigned NPIV-related WWNs), which are used to communicate with physical HBAs.
Therefore, virtual machines can utilize up to 4 physical HBAs for NPIV purposes.
Note that HBAs that are not NPIV-aware are skipped in this discovery process because
VPORTs cannot be instantiated on them.
Module 7: Virtualization
When designing a NPIV solution for virtual machines, be aware of the following requirements
and limitations.
The physical HBAs on an ESX Server host, using their own WWNs, must have access to
all LUNs that are to be accessed by virtual machines running on that host.
NPIV can only be used for virtual machines with RDM disks. Virtual machines with
regular virtual disks use the WWNs of the hosts physical HBAs.
RDMs are like symbolic link from VMFS Volume to raw LUNs. When you map a LUN
to VMFS, it creates a file with extension vmdk, which points to RAW LUN. Basically
data is written on the physical disk.
The ESX Server hosts physical HBAs must support NPIV. Currently, the following
vendors and types of HBA provide this support:
QLogic any 4 Gbps or 8 Gbps HBA.
Emulex - 4 Gbps HBA running firmware level 2.70a5 or later. All Emulex 8 Gbps
HBAs running firmware 1.00a9 or later
Module 7: Virtualization
Module 7: Virtualization
In order to configure NPIV in a virtual machine, it must be turned off. The first thing to do is
verify the physical HBA is capable of supporting NPIV. Its safe to say 4 Gbps HBAs are NPIV
capable as long as they have the correct EMC-recommended firmware.
Verify that LUNs for VMFS datastores and RDM disk are already created, zoned, and LUNmasked to the vSphere physical HBA ports. In the vSphere client:
Select the physical host > Configuration > Storage Adapters
Select the Fibre Channel port to display the available LUNs in the Details section.
At least one VMFS datastore must be created. This is required because the pointer to
the RDM datastore resides in VMFS. With an RDM, a physical LUN is presented to a
VM as a .vmdk-file. From an ESX Server perspective, the VM is still accessing the
vmdk-file, even though this file is actually a pointer that redirects all SCSI traffic to the
raw LUN.
Once you have confirmed that the host has the correct firmware and there is at least one
free RDM disk, you can enable NPIV. To assign a WWN to an existing VM using Virtual Center:
Right-click the virtual machine and select Edit Settings.
Select Options tab and click Fibre Channel NPIV.
Select Generate new WWNs and select the number of WWNNs and WWPNs. In
most cases, if a dual fabric is implemented with an Active/Active array, then two
WWNNs and two WWPNs should be enough.
If an RDM disk has not been assigned to the VM, then add it
The WWNs assigned in this step are used when zoning and LUN masking your virtual
machine.
Copyright 2013 EMC Corporation. All rights reserved
Module 7: Virtualization
For most B-Series switches and directors NPIV is enabled for every port. To enable or disable
NPIV on a port-by-port basis, enter the portCfgNPIVPort command. If the NPIV feature is
disabled, the port is toggled if NPIV devices are logged in from that F_Port (a true NPIV port).
Otherwise the firmware considers that port as an F_Port even though the NPIV feature was
enabled. The example above shows NPIV being enabled on port 15.
Once NPIV is enabled on the port, you can specify the number of logins per port. The
number of virtual N_Port_IDs can be set to a value between 1 through 255 per port. The
default setting is 126. The limit of NPIV logins can be also limited by the addressing mode to
127 or 63 depending on the mode. The portCfgNPIVPort used to enable NPIV, can be also
used to set the maximum number of NPIV logins. The port must be disabled to change this
setting. The example above shows how to set the NPIV logins on port 15.
Use the portCfgShow command to verify the parameters.
Check the EMC Support Matrix to verify the number of NPIV devices supported for a specific
B-Series model.
Module 7: Virtualization
Use the switchShow and portShow commands to view NPIV information for a given port. If a
port is an F_Port, and you enter the switchShow command, then the port WWN of the
N_Port is returned. For an NPIV F_Port, there are multiple N_Ports, each with a different port
WWN. The switchShow command output indicates whether or not a port is an NPIV F_Port,
and identifies the number of virtual N_Ports behind it.
Module 7: Virtualization
Use the portLoginShow command to display the login information for the virtual PIDs of a
port. The portShow command is also used to view the NPIV attributes and all the N_Port
(physical and virtual) port WWNs that are listed under portWwn of device(s) connected
Module 7: Virtualization
10
Prior to zoning the VM, be sure that you have zoned all the physical HBA WWNs to the
correct storage array port WWN/s. Storage best practices are to zone a single initiator to
single target to maintain security and reduce interference. Now that you have the virtual
WWNs generated by NPIV in VM, you then need to create unique zones for each VM to the
storage array port WWN/s.
Module 7: Virtualization
11
All virtual N_Ports must still belong to the same VSAN. The VSAN is determined by the VSAN
assigned to that physical port, either by port number or by the node WWN of the physical
HBA that is attached to the port.
You must globally enable NPIV for all VSANs on the MDS switch to allow the NPIV-enabled
applications to use multiple N_Port identifiers.
To configure NPIV in MDS-Series switches start by enabling the NPIV feature. Then create a
zone between the physical HBA and the port in your Storage Array. Since the virtual port in
the virtual machine hasnt performed a FLOGI, manually zone your virtual WWN to your
storage. From the storage array manually register the virtual HBA and bind LUNs to both the
physical WWN and the virtual WWN.
Once the steps above are completed, turn on your virtual machine and make sure the port in
the switch comes up as an NPIV public port.
Module 7: Virtualization
12
When configuring MDS-Series switches for NPIV, the first thing to do is enable the NPIV
feature. NPIV can be enabled from the CLI, DCNM or Device Manager. The example above
shows how to enable NPIV from the Feature Control window of Device Manager. Under
Action, select Enable and click Apply.
Module 7: Virtualization
13
Before the virtual WWNs from the virtual machine log into the switch they need to be zoned.
The virtual WWN must be added manually to the zone. From Fabric Manager in the Edit
Local Full Zone Database, right-click the zone and select Insert. Select WWN and type the
WWN of the virtual machine. Then add the storage to the zone. Add your zone to the zone
set and re-activate your zone set.
Module 7: Virtualization
14
Although NPIV is transparent to the storage arrays, some systems such as VNX/CLARiiON
require specific configurations to support NPIV in VMware. Listed above you can find the
requirements to provision VNX storage to an ESX server with NPIV.
Module 7: Virtualization
15
This lesson covered the NPIV feature and requirements to configure it in VMware
environments using Connectrix switches.
Module 7: Virtualization
16
This lesson covers the Cisco NPV and Brocade Access Gateway features.
Module 7: Virtualization
17
N_Port virtualization (NPV) reduces the number of Fibre Channel domain IDs in SANs.
Switches operating in the NPV mode do not join a fabric. They pass traffic between NPV core
switch links and end devices, which eliminates the domain IDs for these edge switches.
When a switch acts as an NPV edge switch, it doesnt perform any fabric services, and
instead forwards all fabric activity (FLOGI, FDISC, Name Server, Zoning, etc.) to the NPV Core
switch. Care should be taken when enabling or disabling NPV, as in order to enter or exit
from NPV mode, the switch will perform a write erase on the switch and reboot.
NPV mode applies to an entire switch. All end devices connected to a switch that is in NPV
mode must log in as an N_Port to use this feature. All links from the edge switches to the
NPV core switches are established as NP_Ports (not E_Ports), which are used for typical ISLs.
NPIV is used by the switches in NPV mode to log in to multiple end devices that share a link
to the NPV core switch.
Module 7: Virtualization
18
NP_Ports
An NP_Port (proxy N_Port) is a port on a device that is in NPV mode and connected to the
NPV core switch using an F_Port. NP_Ports behave like N_Ports except that in addition to
providing N_Port behavior, they also function as proxies for multiple physical N_Ports.
NP Links
An NP link is basically an NPIV uplink to a specific end device. NP links are established when
the uplink to the NPV core switch comes up; the links are terminated when the uplink goes
down. Once the uplink is established, the NPV switch performs an internal FLOGI to the NPV
core switch, and then (if the FLOGI is successful) registers itself with the NPV core switchs
name server. Subsequent FLOGIs from end devices in this NP link are converted to FDISCs.
Server links are uniformly distributed across the NP links. All the end devices behind a server
link will be mapped to only one NP link.
Module 7: Virtualization
19
When NPV mode is enabled, the switch becomes a transparent proxy that does not
participate in the SAN fabric services, and it can aggregate all the initiators and targets
directed toward the SAN fabric core as if it were a simple multipoint link. NPV is a powerful
tool for scaling the SAN beyond the port density of traditional Fibre Channel switches.
Different NPV devices can be isolated by connecting them to different VSANs. The NPV
device doesnt run Domain Manager, FSPF, Zone Server and Fabric Login Server. The
switching capabilities are disabled and only routing is performed. The NPV functionality
doesnt require a license in the NEX.
The N-Port Virtualizer feature allows transparent connectivity to any core switch that
supports the NPIV feature. If the core switch follows the standard NPIV implementation, then
the interoperability of different switch vendors is no longer a concern.
NPV is supported by the following MDS-series switches:
Module 7: Virtualization
20
It is possible to configure zoning for end devices that are connected to NPV devices using all
available member types on the NPV core switch. If fWWN, sWWN, domain, or port-based
zoning is used, then fWWN, sWWN or the domain/port of the NPV core switch should be
used.
Although fWWN-based zoning is supported for NPV devices, it is not recommended because:
Zoning is not enforced at the NPV device (rather, it is enforced on the NPV core
switch).
Multiple devices behind an NPV device log in via the same F port on the core (hence,
they use same fWWN and cannot be separated into different zones).
The same device might log in using different fWWNs on the core switch (depending
on the NPV link it uses) and may need to be zoned using different fWWNs.
NPV uses a load balancing algorithm to automatically assign end devices in a VSAN to one of
the NPV core switch links (in the same VSAN) upon initial login. If there are multiple NPV
core switch links in the same VSAN, then you cannot assign a specific one to an end device.
By grouping devices into different NPV sessions based on VSANs, it is possible to support
multiple VSANs on the NPV-enabled switch. The correct uplink must be selected based on
the VSAN that the uplink is carrying.
Module 7: Virtualization
21
Module 7: Virtualization
22
Module 7: Virtualization
23
The NPV mode used on MDS-Series is somewhat similar to Brocade Access Gateway. Access
Gateway is a Fabric OS feature that permits configure an Enterprise fabric to handle
additional N_Ports instead of domains. To do this, configure F_Ports to connect to the fabric
as N_Ports, which increases the number of device ports you can connect to a single fabric.
Multiple Access Gateways can connect to the ED-DCX-B enterprise-class platform, directors,
and switches. After set a B-Series switch to AG mode, the F_Ports connect to the Enterprise
fabric as N_Ports rather than as E_Ports. They connect as E_Ports if the B-Series switch is in
Native mode.
B-Series switches in AG mode are logically transparent to the host and the fabric. You can
increase the number of hosts to have access to the fabric without increasing the number of
switches. This simplifies configuration and management in a large fabric by reducing the
number of domain IDs and ports.
The B-Series switch in Native mode is a part of the fabric. It requires two to four times as
many physical ports, consumes fabric resources, and can connect to a B-Series or M-Model
fabric only. AG is outside the fabric. It reduces the number of switches in the fabric and the
number of required physical ports. You can connect AG to either a B-Series, M-EOS, or Ciscobased fabric.
Module 7: Virtualization
24
Access Gateway multiplexes host connections to the fabric. It presents an F_Port to the host
and an N_Port to an edge fabric switch. Using N_Port ID virtualization (NPIV), AG allows
multiple FC initiators to access the SAN on the same physical port. This reduces the hardware
requirements and management overhead of hosts to the SAN connections.
A fabric switch presents F_Ports (or FL_Ports) and storage devices to the host and presents
E_Ports, VE_Ports, or EX_Ports to other switches in the fabric. A fabric switch consumes SAN
resources, such as domain IDs, and participates in fabric management and zoning
distribution. A fabric switch requires more physical ports than AG to connect the same
number of hosts.
Shown in the slide is a comparison of the types of ports a switch in AG mode uses to the type
of ports that a standard fabric switch uses.
Module 7: Virtualization
25
The B-Series switch in Native mode is a part of the fabric; it requires two to four times as
many physical ports, consumes fabric resources, and can connect to a B-Series or M-Model
fabric only. AG is outside the fabric; it reduces the number of switches in the fabric and the
number of required physical ports. You can connect AG to either a B-Series, M-EOS, or Ciscobased fabric.
When a switch is behaving as an Access Gateway, Role-Based Access Control (RBAC) features
in Fabric OS are available, but Admin Domains, Advanced Performance Monitoring, direct
connection to SAN target devices, Fibre Channel Arbitrated Loop support, Fabric Manager,
FICON, IP over FC, trunking, extended fabrics, management platform services, name services
(SNS), port mirroring, SMI-S, and zoning are not available. ISL is not supported because the
switch is logically transparent to the fabric, therefore it does not participate in the SAN as a
fabric switch.
Module 7: Virtualization
26
The steps for configuring Access Gateway are listed on this slide.
Module 7: Virtualization
27
To connect hosts to the fabric using Access Gateway, configure the fabric using the following
parameters:
Install and configure the switch as described in the switchs hardware reference manual
before performing these procedures.
Verify that the interop mode parameter is set to 0, Brocade Native mode, or the switch mode
is in Native mode.
Configure the F_Ports on the switch to which Access Gateway is connected as follows:
1. Enable NPIV.
2. Disable long distance mode.
3. Allow multiple logins. The recommended fabric login setting is the maximum allowed
per port and per switch.
Use only WWN zoning throughout the fabric. Access Gateway does not support domain ID
and other types of zoning schemes.
Include the Access Gateway WWN or the port WWN of the N_Ports, also include the HBA
WWNs that will be connected to AG F_Ports to the ACL list in ACL policies.
Allow inband queries for forwarded fabric management requests from the hosts. Add the
Access Gateway switch WWN to the access list if inband queries are restricted.
Module 7: Virtualization
28
Before enabling a switch to AG mode, you must save the switch configuration because after
you enable AG mode, some fabric information is erased, such as the zone and security
databases. Enabling AG mode is disruptive; the switch is disabled and rebooted.
If you are setting the DS-300B switch to AG modes, you must enable all ports using POD
licensing before enabling Access Gateway mode.
Ensure that no zoning or AD transaction buffers are active. If any transaction buffer is active,
enabling AG mode will fail with the error, Failed to clear Zoning/Admin Domain
configuration.
Enter the ag --modeenable command.
The switch automatically reboots and comes back online in AG mode using a factory default
F_Port to N_Port mapping.
To verify if Access Gateway mode is enabled, enter the ag --modeshow command.
Module 7: Virtualization
29
Enter the ag --mapshow command without any options to display all the mapped ports.
The ag --mapshow command shows all the N_Ports (with the portcfgnport value of 1) even if
those N_Ports are not connected.
Module 7: Virtualization
30
To display switches in access gateway mode, run the following command from CLI:
agshow
Module 7: Virtualization
31
This lesson covered the Cisco NPV and Brocade Access Gateway features.
Module 7: Virtualization
32
This lab assigns storage through NPIV to virtual machines in an ESXi host
Module 7: Virtualization
33
This module covered the virtualization features offered by the Connectrix switches, including
NPIV, NPV and Access Gateway.
Module 7: Virtualization
34
This module focuses on the concept of transforming a SAN to the Cloud environment.
This lesson covers the subjects that must be understood when considering the
transformation to a Cloud infrastructure.
With all of the interest surrounding Cloud, it is helpful to understand what is driving this need
to change. Each IT organization has its own unique drivers, but they generally fall into some
general categories: cost, availability, time-to-market, etc. There are pressures outside IT from
the organizations highest level executives who are looking for more flexibility, doing more
with less cost, and using information as a competitive advantage. IT organizations want to
respond by transforming IT into something with greater business agility. The IT challenges
listed below have made organizations to think about Cloud Computing model to provide
better service to their customers:
Globalization: IT must meet the business needs to serve the customers world-wide,
round the clock 24x7x365
Cost of ownership: Due to increasing business demand, the cost of buying new
equipments, power, cooling, support, licenses, etc. increases the Total Cost of
Ownership (TCO)
An infrastructure should fulfill the essential characteristics to provide Cloud services. It can
be built using shared pool of computing resources such as compute, storage, and network.
The infrastructure should be flexible to meet rapidly changing demands of its consumers and
allow them to provision resources on-demand over a network. The infrastructure should also
enable monitoring , control and optimization of resource usage.
Building Cloud infrastructure is a phased approach. The journey begins with understanding
the existing physical infrastructure, its elements and processes. The next step is to focus on
aggregating existing infrastructure resources using virtualization technologies. These
resource pools facilitate centralized management of resources and enables faster resource
provisioning.
The next step is to deploy service management tools that enables automation of processes
and management to minimize human intervention. Service management tools also include
measured services which enables consumption based metering. With the service
management in place, on-demand provisioning of IT resources become more dynamic and
allow IT to be delivered as a service.
Although virtualization is a key step towards building Cloud, it is possible to use highly
automated physical infrastructure to provide Cloud services, however it may not be
optimized.
A Classic Data Center (CDC) is a facility that provides IT resources to process data. The core
elements of CDC are compute, storage, network, application, and Database Management
System (DBMS).
Application is a computer program that provides the logic for computing operations.
Applications may use a DBMS, which uses operating system services to perform
store/retrieve operations on storage devices.
DBMS provides a structured way to store data in logically organized tables that are
interrelated. A DBMS optimizes the storage and retrieval of data.
Compute is a resource that runs applications with the help of underlying computing
components.
Storage is a resource that stores data persistently for subsequent use.
Network is a data path that facilitates communication between compute systems or
between compute systems and storage.
These IT resources are typically viewed and managed as separate entities. But, all these
elements must work together to address data processing requirements. Other elements of a
CDC are power supplies and environmental controls such as air conditioning and fire
suppression.
Virtualization abstracts physical resources such as compute, storage, and network to function
as logical resources. It creates an abstraction layer to hide the physical characteristics of
resources from users. For example, in compute system virtualization, a physical machine
appears as multiple logical machines (virtual machines), each running an operating system
concurrently.
A VDC is a data center in which the compute, storage, network, and/or applications are
virtualized. Compute virtualization enables running multiple operating systems concurrently
on a compute system. This improves compute system utilization. Storage virtualization
provides logical view of storage and presents it to the compute system. In network
virtualization, multiple logical networks are created on a physical network. Each of these
virtualization technologies is explained in detail in the forthcoming modules.
By consolidating IT resources using virtualization techniques, organizations can optimize their
infrastructure utilization. By improving the utilization of IT assets, organizations can reduce
the costs associated with purchasing new hardware. They also reduce space and energy
costs associated with maintaining the resources. Moreover, less people are required to
administer these resources which further lowers the cost. Virtual resources are created using
software which enables faster deployment compared to deploying physical resources.
Virtualization increases flexibility by allowing to create and reclaim the logical resources
based on business requirements.
Service management tool enables creation and optimization of Cloud services to meet
business objectives and provide value to the consumers. Services built are provided in a
service catalog that allows consumers to choose desired services. Service management
automates service creation and provisioning without any manual intervention. It also
provide monitoring and metering services to measure resource usage and chargeback.
Service management tools are also responsible for managing both physical and virtual
resources that are used to create Cloud services. Example of management activities are
capacity management, configuration management, change management etc. These
management processes enable meeting service assurance and compliance requirements
This lesson covered the subjects that must be understood when considering the
transformation to a Cloud infrastructure.
This lesson covers the definition and components, and the benefits and challenges, of a
Cloud environment.
10
Historically, Cloud Computing has evolved through grid computing, utility computing,
virtualization, service oriented architecture.
Grid computing: It is a form of distributed computing which applies the resources of
numerous heterogeneous computers in a network to work on a single complex task
at the same time. Grid computing enables parallel computing, although its utility is
best for large workloads.
Utility computing: It is a service provisioning model in which a service provider
makes computing resources available to the customer as needed and charges them
for specific usage rather than a flat rate. The word utility is used to make an analogy
to other services, such as water, electrical power that seek to meet fluctuating
customer needs and charge for the resources based on usage rather than on a flatrate basis.
Virtualization: The conversion of traditional computing environments to what is
called a virtualized environment has also accelerated the movement to Cloud
Computing. Virtualizing a computing environment means that the various hardware
and the software resources are viewed and managed as a pool, which provides
improved utilization of resources. The objectives of virtualization are to centralize
management, optimization of resources by over subscription, and use the available
computing capacity as efficiently as possible among the users and applications.
Service Oriented Architecture (SOA): An architectural approach in which applications
make use of services available in the network. Each service provides a specific
function, for example a business function such as payroll tax calculation, processing
purchase order. A deployed SOA- based architecture provides a set of services that
can be used in multiple business domains.
Copyright 2013 EMC Corporation. All rights reserved
11
12
The on-demand and self-service aspects of Cloud Computing mean that a consumer can use
Cloud services as needed without any human intervention with the Cloud service provider.
Consumers can easily get Cloud services by simply requesting the amount of IT resources
required from the service catalog by using self service interface. In order to be effective and
acceptable to the consumer, the self-service interface must be user-friendly.
13
Cloud services are accessed via the network, usually the internet, from broad range of client
platforms such as desktop computer, laptop, mobile phone and thin client. Traditionally,
software like Microsoft Word or Microsoft PowerPoint has been offered as client-based
software. Users have to install the software on their computers in order to use this software
application. It is not possible to access this software in case the user is away from the
computer, where this software is installed. Today, much of the software used can be
accessed over the internet. For example, Google Docs, a web-based document creator and
editor allows user to access and edit documents from any device with an internet
connection, eliminating the need to have access to a particular client platform to edit
documents.
14
Cloud must have a large and flexible resource pool to meet the consumers needs, provide
economies of scale, and meet service level requirements. The resources (compute, storage,
and network) from the pool are dynamically assigned to multiple consumers based on multitenant model. Multitenancy refers to an architecture and design by which multiple
independent clients (tenants) are serviced using single set of resources. In a Cloud, a client
(tenant) could be a user, a user group, or an organization/company. Multitenancy enables
compute, storage, and network resources to be shared among multiple clients. Virtualization
provides ways for enabling multitenancy in Cloud. For example, multiple VMs from different
clients can run simultaneously on the same server with the hypervisor support.
There is a sense of location independence, in that the consumer generally has no knowledge
over the exact location of the provided resources.
15
Rapid elasticity refers to the ability of the Cloud to expand or reduce allocated IT resources
quickly and efficiently. This allocation might be done automatically without any service
interruption. Consumers will take advantage of Cloud when they have large fluctuation in
their IT resource usage. For example, the organization may require to double the number of
web and application servers for the duration of a specific task. They do not want to pay for
the capital expense of having dormant (idle) servers on the floor most of the time and also
want to release these server resources once the task is completed. The Cloud enables to
grow and shrink these resources dynamically and allows the organizations to pay on a usage
basis.
16
Metered service provides billing and chargeback information for the Cloud resource used by
the consumer. The metered services continuously monitors resource usage (CPU time,
bandwidth, storage capacity) and reports to the consumer. Metered services enable
transforming capital expenditure (CAPEX) into pay as you use operational cost.
17
Infrastructure-as-a-Service (IaaS) is the base layer of the Cloud stack. It serves as the
foundation for other two layers (SaaS, PaaS), for their execution. The Cloud infrastructure
such as servers, routers, storage, and other networking components are provided by the IaaS
provider. The consumer hires these resources as a service based on needs and pays only for
the usage. The consumer is able to deploy and run any software, which may include
Operating Systems (OSs) and applications. The consumer does not manage or control the
underlying Cloud infrastructure but has control over OSs, deployed applications. Here,
consumer needs to know the resource requirements for the specific application to exploit
IaaS well. Scaling and elasticity are the responsibilities of the consumer, not the provider. In
fact, IaaS is a mini do-it-yourself data center that you need to configure the resources (server,
storage) to get the job done.
Amazon EC2 is an example of infrastructure as a service. It allows user to build virtual
machines and install their software on top of it. EMC Atmos is the first multi-petabyte
information management offering, designed to help customers automatically manage and
optimize the delivery of rich, unstructured information across large-scale global cloud
storage environments.
18
19
SaaS is the top most layer of the Cloud Computing stack which is directly consumed by end
user. It is the capability provided to the consumer to use the service providers applications
running on a Cloud infrastructure and accessible from various client devices through a thin
client interface such as a web browser. On-premise applications are quite expensive and
requires high upfront CAPEX (Capital Expenditure). They also incur significant administration
cost. In SaaS model, the applications like Customer Relationship Management (CRM), Email,
Instant Messaging (IM) are offered as a service by a Cloud provider. Here, the consumers
will use only the applications they really want and pay subscription fee for the usage. The
Cloud provider hosts and manages the required infrastructure and applications to support
these services.
SaaS offers following advantages:
Reduces the need for infrastructure because storage and compute power can be
provided remotely
Reduces the need for manual updates because SaaS providers can perform those
tasks automatically
A few leading examples of software as a service are Salesforce.com, which provides access to
the various business software to its subscribers, EMC Mozy, which offers secured online
backup service to its clients, and Google Apps, offering email and word processing services.
20
Cloud Computing can be classified into three deployment models, private, public, and hybrid,
which provide a basis for how Cloud infrastructures are constructed and consumed.
In a Public Cloud, IT resources are made available to the general public or organizations and
owned by Cloud service provider. The Cloud services are accessible to everyone via standard
internet connections. In a public Cloud, a service provider makes IT resources such as
applications, storage capacity, or server compute cycles available to any consumer. This
model can be thought of as an on-demand and a pay-as-you-go environment where
there are no on-site infrastructure or management requirements. However, for organizations,
these benefits come with certain risks: no control over the resources in the cloud, the
security of confidential data, network performance issues, and interoperability. Popular
examples of public clouds include Amazons Elastic Compute Cloud (EC2), Google Apps, and
Salesforce.com.
21
In private Cloud, the Cloud infrastructure is operated solely for one organization and is not
shared with other organizations. This Cloud model offers the greatest level of security and
control. There are two variations to a private Cloud:
On-premise Private Cloud: On-premise private Clouds, also known as internal Clouds
are hosted by organization within their own data centers. This model provides a more
standardized process and protection, but is limited in terms of size and scalability.
Organizations would also need to incur the capital and operational costs for the
physical resources. This is best suited for applications which require complete control
and configurability of the infrastructure and security.
Externally hosted Private Cloud: This type of private Cloud is hosted externally with a
Cloud provider, where the provider facilitates an exclusive Cloud environment for a
specific organization with full guarantee of privacy or confidentiality. This is best
suited for organizations that do not prefer a public Cloud due to data privacy/security
concerns.
Like a public Cloud, a private Cloud also makes provisioning an automated service request
rather than a manual task processed by IT. In on-premise private Cloud, organizations will
have to run their own hardware, storage, networking, hypervisor, and Cloud software. Many
enterprises now offer Cloud platforms to build a private Cloud, including EMC, Cisco, IBM,
Microsoft, Oracle, and VMware, as well as services to manage it.
22
In hybrid Cloud environment, the organization consumes resources from both private Cloud
and public Cloud. The ability to augment a private Cloud with the resources of a public Cloud
can be used to maintain service levels in the face of rapid workload fluctuations.
Organization use their computing resources on private Cloud for normal usage, but access
the public Cloud for high/peak load requirements and this ensures that a sudden increase in
computing requirement is handled gracefully. For example, an organization might use a
public Cloud service such as Amazon Simple Storage Service (Amazon S3) for archiving data
but continue to maintain in-house storage for operational customer data. Ideally, the hybrid
approach allows a business to take advantage of the scalability and cost-effectiveness that a
public Cloud Computing environment offers without exposing mission-critical applications
and data to third-party vulnerabilities.
23
24
Cloud computing has changed the economics of IT. Capital expenditure (CAPEX) is required to
build IT infrastructure. As organizations hire and use resources from Cloud service providers,
they will see more of Operational Expenditure (OPEX). The Cloud provides various cost
savings:
Power and Energy cost: As energy costs continue to rise, power consumption has
become a concern for most organizations. The organizations that use Cloud
applications and services save on power and energy use. An increase in energy
efficiency translates into smaller carbon footprints for organizations, making Cloud a
greener solution than traditional on-premise models.
Note:
CAPEX: A capital expenditure which is incurred to buy fixed assets, for examples
servers, storage, etc.
25
Reduced IT cost: Cloud services can be hired. Therefore, consumers can save money as there
is no capital expenditure or CAPEX required. Consumers can leverage Cloud service
providers infrastructure. Hence, there is no ongoing expenses for running datacenter such as
cost of power and cooling, management, and additionally the real estate cost can be
minimized.
Support business agility: The speed at which new computing capacity can be provisioned is a
vital element of Cloud Computing. Cloud can reduce the time needed to provision and
deploy new applications and services from months to minutes. Cloud allows organizations to
react more quickly to market conditions and enables to scale up and scale down the
resources as needed.
Flexible scaling: Cloud can be easily and instantly scaled up and scaled down based on
demand. It appears to the consumers that Cloud resources are expandable to infinite limit.
Cloud service users can independently and automatically scale their computing capabilities
without any interaction with Cloud service providers.
High Availability: Cloud Computing has the ability to ensure application availability at varying
levels depending on customer policy and priority of the application. Redundant server,
network resources, and storage equipment along with clustered software enable fault
tolerance for Cloud infrastructure. These techniques encompass multiple datacenters in
different geographic regions having identical resource configuration and application
instances, which prevents data unavailability due to regional failures.
Less Energy Consumption: Going Green is an important focus for many organizations.
Cloud enables organizations to reduce power consumption and space usage.
Copyright 2013 EMC Corporation. All rights reserved
26
Both the Cloud consumers and providers have their own challenges. The following are the
challenges of the consumers:
Security and Regulations: Consumers may have business critical data which requires
protection and continuous monitoring of its access. With the Cloud, the consumer
may lose control of the sensitive data for example, the consumer may not know in
which country the data is being stored and may violate some national data protection
statutes (EU Data Protection Directive and U.S. Safe Harbor program). Many
regulations impose restriction to distribute data outside organizations territory.
Network latency: Consumers may access Cloud services from anywhere in the world.
Although Cloud resources are distributed, the resources may not be close to the
consumer location resulting to high network latency. Higher network latency will
result in application timeout and end users may not be able to access the application.
Supportability: Cloud may not support all applications. For example, a consumer may
want to leverage Cloud platform service for their proprietary applications but Cloud
provider may not have a compatible Operating System (OS). Also, legacy applications
may not be supported in Cloud.
27
Service warranty and service cost: Cloud service providers usually publishes Service
Level Agreement (SLA) so that their consumers know about the availability of service,
quality of service, downtime compensation, and legal and regulatory clauses.
Alternatively, customer specific SLAs may be signed between a Cloud service provider
and a consumer. Cloud providers must ensure that they have adequate resources to
provide required level of services. SLAs typically mention penalty amount, if Cloud
service providers fail to provide services. As the Cloud resources are distributed and
continuously scaled to meet variable demands, it is a challenge to the Cloud providers
to manage physical resources and estimate the actual cost of providing the service.
Number of software to manage: Cloud providers especially SaaS and PaaS providers
manage number of applications, different Operating Systems (OSs), and middleware
software to meet the need of wide range of consumers. This requires service
providers to have enough licenses of various software products, resulting in
unpredictable ROI.
No standard Cloud access interface: Cloud service providers usually offer proprietary
applications to access their Cloud. However, consumers may want open APIs or
standard APIs to become tenant of multiple Clouds. This is a challenge for Cloud
providers because this requires agreement among Cloud providers and upgrade of
their proprietary applications to meet the standard.
28
This lesson covered the definition and components, and the benefits and challenges, of a
Cloud environment.
29
This lesson covers the concepts and considerations when migrating to the Cloud.
30
Organizations are not only looking to get a financial advantage with their first move into
Cloud, but are also making a significant learning experience to expand their Cloud
perspective. Businesses, determining how to make their first move into the Cloud always
face a question How Cloud fits to organizations environment? since there is a risk of
introducing evolving Cloud into an established system.
Most companies are not ready to abandon their existing IT investments to move all of their
business processes fully to the Cloud at once. Instead, it is more likely to be a gradual shift in
business processes to the Cloud over time. The reason behind this cautious approach is that
the Cloud providers are not assuring the same levels of security, controls, and performance
that organizations have on premises. Lack of regulatory compliance and policies for both
providers and consumers further slow down the adoption of Cloud.
It is important to understand various Cloud migration considerations before migrating to the
Cloud.
31
Studies based on the experience of early Cloud adopters suggest that moving to the Cloud
without proper strategy and process does not yield expected benefits. The most important
concern that needs to be evaluated before making a move to Cloud is How Cloud Computing
fits in the context of organizations overall business strategy?. Sometimes a Cloud may look
attractive from an application perspective, but may be a challenge for network administrator
in a real time environment, or security may say no to have data outside the firewall.
Risk versus convenience is a key consideration for deciding Cloud migration strategy and
form the basis for choosing right Cloud model. Cloud benefits are well established, but data
may reside outside the organizations perimeter which involves risk. A balance must be
evaluated to determine how much risk an individual or organization may handle for the
benefit of convenience. This proportion varies among Cloud consumers, based on which they
may be segmented into individual, business startup, small and medium business, and
enterprise. Typically, individuals and startup businesses are ready to take high risk to get
most of the convenience offered by a Cloud. Compared to that, SMBs and enterprises are
more sensitive to risk and are unlikely to move their applications to Cloud.
32
Let us understand which Cloud model will be most suitable for an organization or an
individual.
Public Cloud is preferred by individuals who want to access Cloud services such as
Picasa and Google apps, and are least concerned about the security or availability
risks in Cloud for the most part. Here, cost reduction is the primary objective. Public
Cloud enables the opportunity to access these applications for free or by paying
minimum usage charges.
People who start up businesses from small office or home typically opt for Public
Cloud. A large investment to purchase IT resources is not affordable or may not give
required ROI. Therefore, for obvious reason, convenience offered by the Cloud
outweighs risk.
Small and medium-sized businesses have a moderate customer base and any
anomaly in customer data and service levels may impact their business. Hence, they
may not be willing or be able to put Tier 1 applications such as Online Transaction
Processing (OLTP) in the Cloud. A hybrid Cloud model may fit in this case, which
includes organizations internal IT resources (Private Cloud) and external Public Cloud
resources. Tier 1 application data should never cross the boundary of Private Cloud.
Public Cloud enables cost savings and faster time to market and is typically used for
tier 2, tier 3, and tier 4 applications such as backup, archive, and testing.
Enterprises typically have strong customer base worldwide. The priority is to maintain
critical customer data and service levels with strict enforcement of security policies.
They are highly concerned with the risk and information access control in Cloud. They
are financially capable of building a massive Private Cloud. Many enterprises may not
even want to move any of their applications to Cloud.
Copyright 2013 EMC Corporation. All rights reserved
33
Not all applications are good candidates for Cloud, although it may depend on the capability
of Cloud infrastructure and the quality of service offered by Cloud providers.
When migrating applications to the Cloud, there are three general considerations that may
be used to determine if the application can move to the Cloud. Proprietary and missioncritical applications are core and essential to the business. Often, they are applications that
provide competitive advantages and are usually designed, developed, and maintained inhouse. Typically, the perceived risk and effort to outsource these systems to the Cloud is
high.
Give close consideration to applications that are non-proprietary but are still mission-critical.
Though the effort to migrate these applications to the Cloud may be minimal, the perceived
risk to the business may be deemed high. If the organization does not have adequate
resources to maintain the application or the cost to maintain the application is high, then this
may outweigh the risks.
The sweet spot for migrating applications to the Cloud is the non-proprietary and nonmission critical applications if they are not performance sensitive. These applications have
good compatibility, standardized functionality, and interfaces, making the level of migration
effort minimal in comparison to proprietary applications. Since these are non-proprietary
and non-mission critical application, moving to the Cloud poses minimum risk.
34
Cloud is an emerging technology and many Cloud players are just entering the market. Out of
several Cloud service provider, selecting a provider is a critical task. Some key provider
selection criteria are listed on this slide.
35
As consumers move towards Cloud, the quality and reliability of the services become
important considerations. However the demands of the consumers vary significantly. It is not
possible to fulfill all consumer expectations from the service providers perspective and
hence a balance needs to be made via a negotiation process. At the end of the negotiation
process, provider and consumer commit to an agreement. This agreement is referred to as
Service Level Agreement (SLA). This SLA serves as the foundation for the expected level of
service between the consumer and the provider. The QoS attributes are generally part of an
SLA (such as response time and throughput). However, these attributes change constantly,
and to enforce the agreement, these parameters need to be closely monitored.
Strong Service Level Agreements (SLAs) from Cloud vendors are a must to ensure QoS.
Without these agreements, and penalties for failing to meet them, vendors have less
incentive to maintain performance at the highest levels. SLAs can include factors such as
network availability, performance, etc.
36
There are two key factors that impact Cloud performance; infrastructure performance and
network latency.
Network latency: Performance related to network latency typically arises due to large
data sets being sent to and from the Cloud provider. The larger the dataset, the more
likely the network performance issues come into play.
37
After identifying the right application for the Cloud, it may be moved to the Cloud.
Organizations typically adopt the Cloud in phases for smooth transition. Adoption process
typically consists of four phases. They are assessment phase, proof of concept phase,
migration phase, and optimization phase.
The first phase in Cloud adoption is the assessment phase. To ensure successful assessment,
it is important to define and understand its objectives. Assessment involves consideration of
various factors. Assessment should be performed for each application that is identified as
potential candidate for Cloud. Other key assessments are: financial assessment, security and
compliance assessment, technical assessment, and assessment of issues related to migration
of licensed products.
38
After thorough assessment, identifying the right candidate for the Cloud, and estimating the
efforts required for migration, it is time to test the application with a proof of concept. This
phase helps to understand what an application can do and cannot do in Cloud.
The goal of this phase is to check whether an application runs as expected after migrating it
to the Cloud. It is recommended to do thorough testing of the application during this phase.
In this phase, organization can validate the Cloud technology, test legacy software in the
Cloud, perform necessary benchmarks and set expectations.
This assessment enables organizations to:
39
In this phase, application is migrated to the Cloud. There are two application migration
strategies:
Forklift Migration Strategy: In this strategy, rather than moving applications in parts
over time, all applications are picked up at once and moved to the Cloud. Tightly
coupled applications (multiple applications that are dependent on each other and
cannot be separated) or self-contained applications might be better served by using
the forklift approach. Self-contained web applications that can be treated as single
entity and backup/archival systems are examples of systems that can be moved into
the Cloud using this strategy.
Hybrid Migration Strategy: In this strategy, some parts of the application are moved
to the Cloud while leaving other parts of the application in place. The hybrid
migration strategy can be a low-risk approach to migration of applications to the
Cloud. Rather than moving the entire application at once, parts can be moved and
optimized one at a time. This strategy is good for large systems that involve several
applications and are not tightly coupled.
40
After migrating the application to the Cloud, run the necessary tests and confirm that
everything is working as expected. In this phase, focus on how to optimize the Cloud based
application in order to increase cost savings.
Understand the usage pattern to optimize the resources consumed. To understand the usage
pattern, monitor the resources consumed and the workload. Based on the workload,
resources can be scaled up or scaled down. For example, if a customer-facing website,
deployed on Cloud infrastructure, does not expect any traffic from certain part of the world
in certain time of the day, the resources consumed by that region may be scaled down for
that time.
Inspect the system logs periodically to understand the usage of the resources. Relinquish the
idle resource.
41
This lesson covered the concepts and considerations when migrating to the Cloud.
42
This module covered the concept of transforming a SAN to the Cloud environment.
43
44
This module focuses on SAN troubleshooting and monitoring using SAN management tools.
This lesson covers the most common problem areas in a SAN environment, also the tools and
references used for SAN troubleshooting.
Troubleshooting should begin at the center of the SAN the fabric. Because switches are
located between the hosts and storage devices and have visibility into both sides of the
storage network, starting with them can help narrow the search path. After eliminating the
possibility of a fault within the fabric, see if the problem is on the host side or the storage
side, and continue a more detailed diagnosis from there. Using this approach can quickly
pinpoint and isolate problems.
For example, if a host cannot detect a storage device, run a switch command such as
switchShow for B-Series or show flogi database for MDS-Series to determine if the storage
device is logically connected to the switch. If not, focus first on the switch directly connecting
to storage. Use a vendor-supplied storage diagnostic tools to better understand why it is not
visible to the switch. If the storage can be detected by the switch, and the host still cannot
detect the storage device, then there is still a problem between the host and switch.
These steps provide an overview on how to troubleshoot an environment:
The table above describes the most common problems on each of the problem areas, as well
as the tools used to troubleshoot this problems.
The fabric should be the first place to look for the problem. The most common problems are
missing devices, marginal or degraded links and incorrect configurations for the switch or
zoning in particular. The switch LEDs are a physical indication that something is wrong. The
management software usually provides good monitoring tools.
Storage can have physical issues between the switch and the storage array such as bad cables
or devices. Also, incorrect configurations can be a problem. Once again LEDs are a good
indicator, as well as storage diagnostic tools and switch diagnostic commands.
Most of the problems present in a host are HBA-related. The firmware can be wrong or the
driver was wrongfully installed or configured. To analyze host problems use the host OS
diagnostic tools, as well as driver diagnostic tools and switch diagnostics.
Management applications can be installed or configured incorrectly. The application-specific
tools and resources can be used to troubleshoot.
The first place you should look into when searching for Connectrix documentation is on
Powerlink. The Connectrix family location stores several documents, such as press releases,
articles, competition, data sheets, sales presentations, installation guides, admin guides,
release notes, white papers, etc.
Software such as Data Center Network Manager and Connectrix Manager Converged
Network Edition are located under the Support > Software Downloads > Downloads C link.
By accessing the documentation library for Connectrix, one of the most important
documents to read is the Release Notes for a given product release.
Specific technical notes such as Remote Hardware Support: A Detailed Review Technical
Notes can be found in the same section under Technical Notes/Troubleshooting sub-section.
Connectrix white papers are also available here.
Firmware is not available on Powerlink.
As always, please refer to the current EMC Support Matrix (ESM) and/or check the E-Lab
Interoperability Navigator in order to verify Connectrix supportability and interoperability.
A link to E-Lab Navigator/ESM is available on the main/home page of Powerlink.
Advanced Query This tab allows you to use E-Lab Interoperability Navigator's
standard tree-structured query, which provides flexible access to E-Lab
Interoperability Navigator's interoperability information.
Wizards This tab provides guided search wizards designed to retrieve commonly
used interoperability information.
PDFs and Guides This tab opens the PDFs and Guides page. This page contains
downloadable PDF and .zip files for multiple EMC Support Matrix versions (targeted
to specific hosts, storage families, or operating systems), Host Connectivity Guides,
Software Support documents, and other documentation you may need.
Two master articles exist which point to current hot spots in support:
Another useful set of documents when it comes to EMC networked storage products, are the
EMC TechBooks. These are a series of Tech books that separate the data in logical sections.
Make sure you have the current versions, as these documents have frequent updates.
10
The EMC knowledgebase search is a very powerful and handy tool when it comes to problem
resolution. Closed cases and their resolutions are available to help people out in the field.
11
12
This lesson covered the most common problem areas in a SAN environment, also the tools
and references used for SAN troubleshooting.
13
This lesson covers SAN Troubleshooting and monitoring using Connectrix Manager
Converged Network Edition.
14
If a host is unable to detect its target, for example, a storage or tape device, you should begin
troubleshooting the problem at the switch. Determine if the problem is the target or the
host, then continue to divide the suspected problem-path in half until you can pinpoint the
problem.
The logical connection is checked to make sure the devices are connecting to the switch. The
switchShow command shows if the hosts and targets are connected. If the devices are
connected they will probably appear as F-Ports. Devices that are not connected will appear
as G-Ports or U-Ports. If the device doesn't appear, verify the configuration, cabling and
physical problems in the initiator and target side.
If the devices appear connected, but the host cannot detect the target, check the name
server with the nsShow command. Verify if both devices are connected to the network. If
the devices are connected to the switch, the problem is between the initiator and target, not
the switch. At this point verify zoning is setup properly. If one or more device is not
connected, the problem is between that device and the switch. The problem can be a
timeout in the communication or a login issue.
Other useful commands for troubleshooting are:
portLoginShow Verify port login status.
15
A correct login is when the port type matches the device type that is plugged in. When
having connectivity problems, use the portCfgShow command to verify port configuration.
In some cases, you may find that the port has been locked as an L_Port and the device
attached is a fabric point-to-point device such as a host or switch. This would be an incorrect
configuration for the device and therefore the device cannot log into the switch. To correct
this type of problem, remove the Lock L_Port configuration using the portCfgDefault
command.
Enter the portErrShow command; then, check for errors that can cause login problems. A
steadily increasing number of errors can indicate a problem. Track errors by sampling the
port errors every five or ten minutes.
16
When a new fabric is discovered the tracking option is automatically enabled and the switch
begins to be monitored. Subsequently, if a switch or end-device is added to or removed from
the fabric, a plus (+) or minus (-) icon displays next to the product icon. Connections are also
tracked. A new connection displays a solid gray line with a added icon and missing
connections display a yellow dashed line with a removed icon.
To enable fabric tracking select a fabric on the Product List or Connectivity Map and select
Monitor > Track Fabric Changes, or right-click a fabric on the Product List or Connectivity
Map and select Track Fabric Changes.
To accept all changes to a fabric:
Select a fabric on the Product List or Connectivity Map and select Monitor > Accept
Changes, or right-click a fabric on the Product List or Connectivity Map and select
Accept Changes.
The Accept Changes Summary dialog box is displayed and it includes information
such as Fabric Name, Switches, Device Ports and Connections.
17
Performance enables you to provision end-to-end monitors of selected target and initiator
pairs. These monitors are persisted in the database and are enabled on one of the F_Ports
(the Management application server determines the port) on the connected switch. You can
use these monitors to view both real time and historical performance data. For end-to-end
monitors to work a fabric OS device must exist in the fabric.
In order to create end-to-end monitoring the initiator or target device should have
Performance Monitor license.
To select the initiator and target to monitor go to:
18
The Device Connectivity Troubleshooting dialog box allows you to select two device ports
from one fabric or two different fabrics and run the diagnostics to test connectivity.
To troubleshoot device connectivity, select Configure > FC Troubleshooting > Device
Connectivity.
Select the source and destination ports on which you want to troubleshoot device
connectivity
Enter the source and destination port FC Address or select the ports directly from the
switches by selecting the Select two device ports option.
Click OK.
The following diagnostic tests are performed:
Device Status
Switch port health status
Zone configuration in the fabric
LSAN zone configuration in edge fabrics
Edge fabric - FC router physical connection status.
Active ACL DCC policy check (Fabric OS only)
19
CMCNE allows to apply thresholds and event notification to real-time performance data. A
process monitors the performance data against the threshold setting for each port and issues
an appropriate alert to notify you when the threshold is exceeded. A threshold policy
manages when to generate events or write to the master log. From the configure threshold
window you can add a policy. Give it a name and description and select which type of device
the policy will apply to. Then configure a high and low threshold. Every time a counter
reaches one of these thresholds an event will appear. Many events can be generated, to
avoid an excess of events in a short amount of time configure a buffer. This would be a range
of values just below the upper boundary and just above the lower boundary where no event
will be triggered. Finally add your threshold policy.
The threshold policy now needs to be assigned to a fabric or device. This can be done from:
20
Performance connection utilization turns the utilization display on and off from the menu
and tool bar. When enabled, it displays moving dotted colored lines that originate from a
port. Two lines are displayed in the topology when turned on; one represents percentage
utilization for transmit and the other percentage utilization for receive. The movement of the
line determines if it is a transmit or a receive.
21
CMCNE provides a variety of logs through which you can monitor the SAN. You can view all
events that take place in the SAN through the Master Log at the bottom of the main window.
You can also view a specific log by selecting an option from the Monitor > Logs.
The logs are described in the following list:
Audit Log: Displays all Application Events raised by the application modules and all Audit
Syslog messages from the switches.
Event Log: Displays all Product Event type events from all discovered switches.
Fabric Log: Displays Product Events Device Status, and Product Audit type events for all
discovered fabrics.
FICON Log: Displays all the LIR and RLIR type events, for example, link incident type events.
Product Status Log Displays events which indicate a change in Switch Status for all
discovered switches.
Security Log: Displays all security events for the discovered switches.
Syslog Log: Displays syslog messages from switches.
22
CMCNE records the SAN events in the Master Log. You can configure the application to send
event notifications to email addresses at certain time intervals. This is a convenient way to
keep track of events that occur on the SAN.
To send notification of events to users, select Monitor > Event Notification > Email
1.
2.
3.
4.
5.
6.
7.
23
You can use Technical Support to collect SupportSave data (such as, RASLOG, TRACE, and so
on) and switch events from Fabric OS devices.
To capture technical support and event information for specified devices, select Monitor >
Technical Support > Switch/Host SupportSave. You must have the SupportSave privilege to
perform this task.
24
The supportSave command save RASLOG, TRACE, supportShow, and other support
information to a FTP server in interactive mode:
1. Connect to the switch through a Telnet or SSH utility or a serial console connection.
2. Log in using an account assigned to the admin role.
3. Type the supportshow command.
When invoked without operands, this command goes into interactive mode.
25
This lesson covered SAN Troubleshooting and monitoring using Connectrix Manager
Converged Network Edition.
26
This lesson covers SAN Troubleshooting and monitoring using Data Center Network Manager.
27
Basic connectivity issues can prevent a host from seeing its target. From the CLI, show the
fibre channel name server database to verify if a port has successfully registered with the
fabric name server. The command in order to do it is: show fcns database.
The FCNS database is also available through device manager by clicking FC > Name Server.
If the host or storage hasnt registered with the name server database, verify it has
performed a FLOGI. Show the FLOGI database with the command:
show flogi database
Make sure the correct pWWN for the HBA and storage array show up on the correct port.
The following questions should be answered when troubleshooting basic connectivity:
Are you using the correct fiber (SM or MM)?
Did you check for a broken fiber?
Is the Fibre Channel port LED on the connected module green, and do the LEDs on
any HBA/storage subsystem ports indicate normal functionality?
Is there a LUN masking policy applied on the storage subsystem? If yes, is the server
allowed to see the LUNs exported by the storage array?
Is there a LUN masking policy configured on the host? Did you enable the server to
see all the LUNs it can access?
If LUN masking software is used, is the hosts pWWN listed in the LUN masking
database?
28
Basic port monitoring using Device Manager begins with the visual display in the Device
View. Port display descriptions include:
Green box: A successful fabric login has occurred; the connection is active.
Red X: A small form-factor pluggable transceiver (SFP) is present but there is no
connection. This could indicate a disconnected or faulty cable, or no active device
connection.
Red box: An FSP is present but fabric login (FLOGI) has failed. Typically a mismatch
iN_Port or fabric parameters with the neighboring device. For example, a port
parameter mismatch would occur if a node device were connected to a port
configured as an E_Port. An example of a fabric parameter mismatch would be
differing timeout values.
29
Device Manager provides an easy tool for monitoring ports on the Cisco MDS-Series
switches. This tool gathers statistics at a configurable interval and displays the results in
tables or charts. These statistics show the performance of the selected port in real-time and
can be used for performance monitoring and troubleshooting. For a selected port, you can
monitor any of a number of statistics including traffic in and out, errors, class 2 traffic, and
FICON data. You can set the polling interval from ten seconds to one hour, and display the
results based on a number of selectable options including absolute value, value per second,
and minimum or maximum value per second.
Device manager provides two performance views, the Summary View tab, and the
configurable monitor option per port.
30
To display additional details about port traffic, use the Device View or Summary View. In
Device View, choose one or more ports, right-click and choose Monitoring from the pop-up
menu. In Summary View, choose one or more interfaces, and click the Monitor tool. The
initial display shows traffic information for the selected interval, including the number of
bytes and frames received and transmitted.
Additional tabs include:
Protocol - View protocol-related traffic and error statistics, including link reset counts, offline
and non-operational sequence errors, reset protocol errors, and statistics related to bufferto-buffer flow control.
Discards - View the number of frames discarded by the port, including Class 2, Class 3, and
Class F frames, EISL frames, and totals.
Link Errors - View the number of link errors, including link failures, signal losses,
synchronization failures, invalid transmission words, and delimiter and address identifier
errors.
Frame Errors - View frame error statistics, including the number of frames with invalid CRC,
Class 3 frames that were discarded upon reception, FBSY returns for selected situations, and
FRJT returns resulting from frame rejection by fabric.
Class 2 Traffic - View the amount of Class 2 traffic for the selected interval.
FICON - View FICON error statistics, including pacing, disparity, EOF, OOF, and order sets
errors.
Class 2 Errors - View error statistics for Class 2 traffic, including busy frame responses and
port rejects.
31
You can configure DCNM to gather ISL statistics in real time. These ISL statistics include
receive and transmit utilization, bytes per second, as well as errors and discards per ISL.
This can be done by selecting Performance --> ISLs in Real-Time. The ISL Real-Time
Performance window allows you to select the Poll Interval and bandwidth thresholds.
ISLs statistics gathered are:
Rx Utilization
Tx Utilization
Bytes per second
32
Fabric Manager and Device Manager present concise views of the generated system
messages and other logged events:
Learn to use Threshold Manager to alert you that critical statistics have exceeded a
set threshold
33
You can use the End to End Connectivity option to determine connectivity and routes among
devices with the switch fabric. The connectivity tool checks to see that every pair of end
devices can talk to each other, using a Ping test and by determining if they are in the same
VSAN or in the same active zone. This option uses versions of the ping and traceroute
commands modified for Fibre Channel networks. The ping and redundancy tests are now
mutually exclusive, you cannot run both at the same time.
1. Choose Tools > Connectivity > End to End Connectivity. The End-to-End Connectivity
Analysis dialog box appears.
2. Select the VSAN whose connectivity will be verified from the VSAN drop-down list.
3. Select whether to perform the analysis for all active zones or for the default zone.
4. Click Ensure that members can communicate to perform a Fibre Channel ping between
the selected endpoints.
5. Identify the number of packets, the size of each packet, and the time out in
milliseconds.
6. Analyze the redundant paths between endpoints by checking the Ensure that
redundant paths exist between members check box.
7. Check the Report errors for check box to see a report of zone and device errors.
8. Click Analyze. The End to End Connectivity Analysis window displays the selected
endpoints including the switch to which each is attached, and the source and target
ports used to connect it.
34
The show tech support command is useful when collecting a large amount of information
about your switch for troubleshooting purposes. The output can be provided to technical
support representatives when reporting a problem.
You can issue a show tech support command from Fabric Manager for one or more switches
in a fabric. The results of each command are written to a text file, one file per switch, in a
directory you specify. You can then view these files using Fabric Manager. You can also save
the Fabric Manager map as a JPG file. The file is saved with the name of the seed switch (for
example, 172.22.94.250.jpg). You can zip up all the files (the show tech support output and
the map file image) and send the resulting zipped file to technical support.
1. Choose Tools > Health > Show Tech Support. You see the Show Tech Support dialog
box.
2. Select the switches for which to view tech support information by checking the
checkboxes for each switch.
3. Set the time-out value.
4. Select the folder where you want the text files (containing the tech support
information) to be written.
5. Check the Save Map check box if you want to save a screenshot of your map as a JPG
file.
6. Check the Compress all files as check box to compress the files into a zip file.
7. Click OK to start issuing the show tech support command to the switches you
specified.
35
In order to save the configuration to a file, simply run the terminal length 0
command before the show-tech support details. This causes the output to be
captured. When the command is done executing, simply save the capture to bootflash by
running the following command:
tac-pac bootflash://showtech.switch1
The file can also be copied from bootflash to FTP on a host. To do this use the following
command:
copy bootflash://showtech.gz ftp://10.127.96.150/showtech_mds1.gz
36
This lesson covered SAN Troubleshooting and monitoring using Data Center Network
Manager.
37
38
This module covered SAN troubleshooting and monitoring using SAN management tools.
39
40