Basic Training on Storage

Technologies

Introduction
& Concepts

June 08 -09, 2009

The storage upgradation
process

•End user demands more disk/storage

space
The storage upgradation
process -2
OPTION 1: Additional disk(s) may be added

Host Bus Disk

Adapter(HBA) drive

SCSI / Fibre channel /

SATA
 The storage upgradation
process -3
OPTION 2: Disk space may be shared from another
system on the network

Network
link

System requiring disk System providing disk

space space
 Creating a file system
To make the new disk usable a filesystem has to
be created on the whole or part of the disk
Disk

One File-
system

Disk
One File-
Partition 1
system
One File-
Partition 2
Partition 3
system
One File-
Partition 4
system
One File-
system
Limitations of a single disk
1. Filesystem size limited to disk capacity

2. No redundancy – if the disk fails all data is

lost
3. Performance restricted to that of a single
disk

The solution:

Redundant Array of Independent Disks

(RAID)
 Redundant Array of
Independent Disks(RAID-0)
RAID-0: Data is striped across all the disks in the
array 1 2 3 4 5 6 7 8
Data
Blocks 9 10 11 12 13 14 15 16

Layout of the data blocks on

Diskthe
1 disks
Disk 2 Disk 3 Disk 4

Block 1 Block 2 Block 3 Block 4

Block 5 Block 6 Block 7 Block 8
Block 9 Block 10 Block 11 Block 12
Block 13 Block 14 Block 15 Block 16
 Redundant Array of
Independent Disks(RAID-1)
RAID-1: Data is mirrored for all the disks in the
array 1 2 3 4 …..
Data
Blocks
Layout of the data blocks on
the disks
Disk 1 Disk 2

Block 1 Block1
Block 2 Block 2
Block 3 Block 3
Block 4 Block 4
..… ..…
 Redundant Array of
Independent Disks(RAID-3)
RAID-3: Data is striped with a single parity disk
for redundancy
Data 1 2 3 4 5 6 7 8

Blocks 9 10 11 12 13 14 15 16

Layout of the data blocks on

Disk 1 the
Diskdisks
2 Disk 3 Disk 4 Parity
Disk
Block 1 Block 2 Block 3 Block 4 Parity
Block 5 Block 6 Block 7 Block 8 Parity
Block 9 Block 10 Block 11 Block 12 Parity
Block 13 Block 14 Block 15 Block 16 Parity
 Redundant Array of
Independent Disks(RAID-5)
RAID-5: Data is striped with distributed parity
blocks
Data 1 2 3 4 5 6 7 8

Blocks 9 10 11 12 13 14 15 16

Layout of the data blocks on

Disk 1 the
Diskdisks
2 Disk 3 Disk 4 Parity
Disk
Block 1 Block 2 Block 3 Block 4 Parity
Block 5 Block 6 Block 7 Parity Block 8
Block 9 Block 10 Parity Block 11 Block 12
Block 13 Parity Block 14 Block 15 Block 16
 Redundant Array of
Independent Disks(RAID-6)
RAID-6: Data is striped with two distributed
parity blocks
Data 1 2 3 4 5 6 7 8

Blocks 9 10 11 12 13 14 15 16

Layout of the data blocks on

the disks
Disk 1 Disk 2 Disk 3 Disk 4 Disk5 Disk6

Block 1 Block 2 Block 3 Block 4 Parity Parity

Block 5 Block 6 Block 7 Parity Parity Block 8
Block 9 Block 10 Parity Parity Block 11 Block 12
Block 13 Parity Parity Block 14 Block 15 Block 16
Independent Disks(RAID-
10)
RAID-10: Data blocks are mirrored and then
striped across disks
Data 1 2 3 4 5 6 7 8

Blocks
Layout of the data blocks on
the disks
Striped

Mirrored Mirrored
Disk 1 Disk 2 Disk 3 Disk 4
Block 1 Block 1 Block 2 Block 2
Block 3 Block 3 Block 4 Block 4
Block 5 Block 5 Block 6 Block 6
Block 7 Block 7 Block 8 Block 8
Independent Disks(RAID-
01)
RAID-01: Data blocks are striped and then
mirrored across disks
Data 1 2 3 4 5 6 7 8

Blocks Layout of the data blocks on

the disks
Mirrored
Striped Striped
Disk 1 Disk 2 Disk 3 Disk 4

Block 1 Block 2 Block 1 Block 2

Block 3 Block 4 Block 3 Block 4
Block 5 Block 6 Block 5 Block 6
Block 7 Block 8 Block 7 Block 8
 Host-based RAID
 Implementation of RAID configurations using
programs on the host system.
 Independent of hardware configuration
 Tightly integrated with host Operating
 System
Each operating system has its own
software RAID implementation:
Solaris –Disk Suite, Volume manager
Linux,IBM – LVM
SGI – XVM
 Third party RAID software also exist e.g
Veritas Volume Manager
Limitations of host-based
RAID
1. Limited Filesystem size

2. All RAID operations are additional

overhead on the CPU
3. Storage space cannot be shared by
multiple systems

The solution:

Use an EXTERNAL RAID-based Storage

Array
 Advantages of Storage
Arrays
1. Huge storage capacities – upto PetaByte
(1015 ) levels
2. Totally self contained -independent of the
host ‘s processing power
3. No wastage of storage space –available
capacity can be distributed among
multiple hosts
4. More reliable – can be configured with no
Single Point of Failure (SPOF)
5. Negligible backup window
6. Remote replication possible for
overcoming disaster situations.
 Hardware RAID – Storage
Arrays
Implementation of RAID using hardware
controllers
RAID Controller
DISK
FE-0 Cache BE-0

LOOP-1
LOOP-0
DISK
Fiber Channel/
iSCSI DISK
RAID
Links Controller
DISK
FE-1 BE-1

Storage Array
Components of a Modular
Storage Array
1. Hard disk drives

2. RAID controller(s) having front-end(FE) &

back-end(BE) ports. FE ports are attached
to the hosts while BE ports are connected
to the Disks
3. Battery-backed Cache
The Hard Disk Drive :
Internals
The Hard Disk Drive :
The Actuator
The Hard Disk Drive :
Platters
The Hard Disk Drive :
Spindle speeds
Spindle Speed Average Latency (Half Typical Current
(RPM) Rotation) (ms) Applications
3,600 8.3 Former standard, now
obsolete

4,200 7.1 Laptops

4,500 6.7 IBM Microdrive, laptops

4,900 6.1 Laptops

5,200 5.8 Obsolete

5,400 5.6 Low-end  IDE/ATA,

laptops

7,200 4.2 High-end IDE/ATA, Low-

end SCSI

10,000 3.0 High-end SCSI

12,000 2.5 High-end SCSI

15,000 2.0 Top-of-the-line SCSI

The Hard Disk Drive :
Access Time
Access Time =

Command Overhead Time

+ Seek Time

+ Settle Time

+ Latency
The Hard Disk Drive :
Command Overhead Time

Command Overhead refers to the time that

elapses from when a command is given to the
hard disk until something actually starts
happening to fulfill the command
The Hard Disk Drive :
Seek Time

The seek time of a hard disk measures the

amount of time required for the read/write
heads to move between tracks over the
surfaces of the platters.
The Hard Disk Drive :
Settle Time

The settle time specification (sometimes

called settling time) refers to the amount of
time required, after the actuator has moved
the head assembly during a seek, for the heads
to stabilize sufficiently for the data to begin to
be read.
The Hard Disk Drive :
Latency

Latency is the time the drive must wait for the

correct sector to come around to where the
read/write heads are located after the actuator
assembly has completed its seek to the correct
track.
The Hard Disk Drive :
Latency(continued)
Spindle Speed Worst-Case Latency (Full Average Latency (Half
(RPM) Rotation) (ms) Rotation)(ms)
3,600 16.7 8.3
4,200 14.2 7.1
4,500 13.3 6.7
4,900 12.2 6.1
5,200 11.5 5.8
5,400 11.1 5.6
7,200 8.3 4.2
10,000 6.0 3.0
12,000 5.0 2.5
15,000 4.0 2.0
 The Hard Disk Drive :
External Interface Types
 Small Computer System Interface (SCSI)

 Fibre Channel (FC)

 Parallel AT Attachment (PATA) (also known

as IDE)

 Serial ATA (SATA)

 Serial Access SCSI (SAS)

 The Hard Disk Drive :
Interface Comparison
Name Raw Bandwidth Transfer
(Mbps) Speeds (MB)
eSATA 3000 300
SATA 300 3000 300
SATA 150 1500 150
PATA 133 1064 133
SAS 300 3000 300
SAS 150 1500 150
Ultra320 SCSI 2560 320
FC over optic fiber 8000 1600
FC over copper 4000 400
 The RAID controller
1. Normally configured in pairs for redundancy
and load balancing
2. RAID controller(s) have front-end(FE) &
back-end(BE) ports. FE ports are attached
to the hosts while BE ports are connected
3. to the Disks
Normally 2- 4 FE and BE ports each are
present per controller
4. Controllers have support for protocols for
connecting to the disks (normally FCAL)
and to the hosts (FCAL, FCSW or iSCSI)
5. LUN masking & Multipathing support are
also implemented in the controller
The Battery-backed Cache
1. This is high-speed memory (upto 16GB) for
holding data read from, or to be written to,
the RAID groups in the disks
2. It can withstand power failure for a limited
period (36-72 hours) due to its battery
backup
3. May be mirrored in some arrays for
reliability/ redundancy
4. Divided into separate Read & Write
caches
5. Cache capacity and implementation is
one of the major criteria in storage vendor
6. selection
Is required for improving read/write
performance
 The Hot Spare
1. This is an additional disk drive in any storage
array configured to work as a standby drive
during normal operation but automatically
replaces any disk that becomes non-
2. operational.
In case of an imminent disk failure the array
controller copies all data in the suspected disk
to the hot spare, marks the hotspare as normal
and the suspected disk as defective and
“ready to be replaced”.
3. Hot spares may be, and normally are, more
than one.
4. In the HP EVA series, there is no specific
hot spare.
 Connectivity:
Storage Area Networks
Host(s) are connected to the storage array(s)
via Storage Area Networks (SAN).
SANs facilitate :
 Block-level access to storage (the host sees
the allocated space as a local physical drive)
 Redundant host-storage paths with load
sharing and automatic failover
 Space in the same storage array distributed
among multiple hosts
Storage Area Networks :
Layers
A SAN consists of 3 layers:
 Storage Layer consisting of all storage arrays
and tape drives
 Fabric layer consisting of Fibre Channel
hubs,switches,directors,routers etc. and all
optical fiber cabling
 Host layer consisting of all the hosts, host bus
adapters(HBAs) including Gigabit Interface
Convertors(GBICs), their drivers and
multipathing software
Storage Area Networks :
Layers(contd.)

Host Layer

FC Switch FC Switch Fabric Layer

Storage Layer
Storage Array
Storage Area Networks :
Protocols
A SAN uses (originally) 2 protocols:
 Fibre Channel Protocol which manages the
transport of SCSI protocol messages between
server and storage across FC network
components

 SCSI Protocol which deals with commands for

communication with the storage array
 Fibre Channel :
Active Components

 Hubs – Any 2 ports can communicate at once

 Switches – Any no. of pairs of ports can
communicate simultaneously
 Directors - A highly reliable switch with
expandable ports
 Bridges, Gateways, DataRouters - For
protocol conversion(like SCSI-FC, iSCSI-FC
etc.)
Fibre Channel :
Cable Types
 Fibre Channel :
LC,SC connectors

Lucent Connector Subscriber

Connector
(LC)
(SC)
Fibre Channel :
Switch

Front view

Rear view
Fibre Channel :
Data Director
 Fibre Channel :
Host Bus Adapter(HBA)
Fibre Channel :
Cabling & Switch
 Fibre Channel :
Topologies

 Point-to-Point Topology
 Arbitrated Loop Topology
 Switched Fabric Topology
 Fibre Channel :
Point-to-Point Topology
 In Point-to-Point Topology, there is a direct
connection between the Storage and the
Host(Server) without any switch or hub
sharing the connection.

 Storage connected in this fashion are also

called Direct Accessed Storage (DAS)
 Fibre Channel :
Direct Attached Storage (DAS)
Storage Arrays directly connected to the host
system
Storage Array
Host
-1
DISK
HB Fiber Channel
FE-0 Cache BE-0
A

LOOP-0

LOOP-1
DISK

DISK
Host RAID
Controller
-2
DISK
HBFiber ChannelFE-1 BE-1
A
 Fibre Channel :
Multipathing Software
 Fibre Channel :
Arbitrated Loop Topology
 In Arbitrated Loop Topology, all devices are
connected in a loop using hubs
 The protocol used in this topology is called
Fibre Channel Arbitrated Loop (FA-AL)
 Only one pair of devices in the loop gets
the entire bandwidth at any point of time.
 A Maximum of 126 devices can be present
in a loop
 Fibre Channel :
Switched Fabric Topology
 A switched fabric is a collection of switches
tied together through inter-switch links to
create a fabric of switches.
 Each device in a fabric has an address
called a World Wide Name (WWN) that’s
hard-coded at the factory.This is a 64-bit
number.
 The protocol used in this topology is called
Fibre Channel -Switched (FC-SW) protocol
 Each pair of independent devices in the
fabric gets the entire bandwidth at any
point of time.
 A Maximum of 64 million devices can be
present in a fabric.
 Fibre Channel :
Basic Fabric Topologies
 Dual switch
 Loop of switches
 Meshed fabric
 Star
 Core-edge
 Fibre Channel :
Dual Switch Topology
 Fibre Channel :
Loop-Of-Switches Topology
 Fibre Channel :
Meshed Fabric Topology
 Fibre Channel :
Dual Meshed Fabric Topology
 Fibre Channel :
Star Fabric Topology
 Fibre Channel :
Star –Ring Hybrid Fabric
Topology
 Fibre Channel :
Core-Edge Topology
 Fibre Channel :
Terminology
 FC – Fibre Channel
 SAN – Storage Area Network using SCSI
blocks over an FC-based network
 Switched Fabric – A collection of one or more
interconnected FC switches
 SAN Fabric - A collection of all individual
Switched Fabrics in a SAN
 NAS – Network Attached Storage using
CIFS(Common Internet File System) in
Windows servers and NFS(Network File
System) in Unix servers to transfer files
 Fibre Channel :
Types of Switch Ports

 G-Port – Global port or unconnected port in a

switch
 E-Port, T-Port – Expansion or Trunk port for
inter-switch linking
 F-Port – A switch (Fabric) port connected to a
host or storage
 FL-Port – A switch (Fabric) port connected to
a hub(Loop)
 N-Port – A Node(on a host or a storage array)
port connected to a switch
 NL-Port – A Node(on a host or a storage
array) port connected to a hub(Loop)
 Fibre Channel :
Fabric Zoning

 Zone – A zone is a bunch of ports or World

Wide Name (WWN) addresses that are
isolated together into a group. Only members
within the group can see each other and
communicate among themselves.
 Zone set – A zone set is a given group of
zones that, when activated,becomes the
current configuration that a particular switch
or fabric uses to enforce the rules of who can
talk to whom. Only ONE zone set can be
active at any given time.
 Zone Alias - A zone alias is just a friendly
 Fibre Channel :
Hard and Soft Zones

 Soft Zones – A soft zone is a zone described

in terms of World Wide Numbers(WWNs), i.e.,
devices connected to the fabric are allowed
or disallowed access to one another
depending on their WWNs and irrespective of
which physical port they are connected to.

 Hard Zone – A hard zone is specified using

combinations of domain id of the switch and
the number of the physical port in use.
 Fibre Channel :
LUN Security

 LUN security is a storage array function to

lock down which WWN(HBA) can see which
LUN on the array.

 Also known as LUN Masking or Selective

Storage Presentation(SSP)

 LUN Security feature may or may not be

supported by the Storage Array.
 Storage Area Networks :
An example
UNIX WIN2K
10:00:00:00:00:00:12:34

10:00:00:00:00:00:87:65

10:00:00:00:00:00:56:78

10:00:00:00:00:00:43:21 FC Switch FC Switch

Fabric-A Fabric-B
50:06:0E:80:00:00:20:B1
FC Switch FC Switch
50:06:0E:80:00:00:20:A2 50:06:0E:80:00:00:20:B2

50:06:0E:80:00:00:20:A1
Controll Controll
er-1 er-2
Storage Array
 Storage Area Networks :
An example (contd.)
Steps for configuring zones :
1. Set Domain ID in all switches in each fabric to make
them unique across fabrics.
2. Create an empty zone set for each fabric.
3. Create aliases for each WWN being attached to each
fabric
4. Create a zone for each HBA being connected to the
fabric having one HBA alias/port and all storage ports
being provided access.
5. Add all zones created above to the zone set
6. Enable the zone set
7. Repeat above steps for other fabric
 Storage Area Network
(SAN)
Storage Arrays connected to several host
systems through a fiber channel(FC) switch
Host
-1
Storage Array
HB
A DISK
FE-0 Cach BE-0
e
Host

LOOP-1
LOOP-0
-2
DISK
FC
HB
A
switch
DISK
RAID
Controller
Host
-3
DISK
FE-1 BE-1
HB
A
Network Attached Storage
(NAS)
Storage Arrays connected to a host in turn
sharing disk space
Host sharing Storage Array
disk space
DISK
Local FE-0 Cach BE-0
e

Area

LOOP-1
LOOP-0
DISK
HB
A
DISK
Netw RAID
ork HB Controller
A
DISK
(LAN) FE-1 BE-1
 SAN with NAS Gateway
Storage Arrays providing both SAN & NAS
connectivity
Host
-1
Storage Array
HB
A DISK
FE-0 Cach BE-0
e
Host

LOOP-1
LOOP-0
-2
DISK
FC
HB
A
switch
DISK
RAID
Controller
Host
-3
DISK
Sharing disk FE-1 BE-1
space
HB
A

LAN
NAS Gateway
Thank you