Technology considerations in selecting a direct attached storage solution for HP ProLiant Gen8 servers

Technology brief
Introduction .........................................................................................................................................2 Interconnect technology.........................................................................................................................2 Serial ATA .......................................................................................................................................2 Serial Attached SCSI .........................................................................................................................2 Interoperability .................................................................................................................................2 Expanders .......................................................................................................................................2 SAS/SATA link speed specifications and performance .......................................................................... 3 Drive controller technology ....................................................................................................................3 Host bus adapters .............................................................................................................................4 Smart Array controllers ......................................................................................................................4 Storage infrastructure ............................................................................................................................5 HP SmartDrive carriers ......................................................................................................................5 Internal expander backplanes ............................................................................................................ 5 BladeSystem direct attached storage ................................................................................................... 6 HP hard disk drive categories ................................................................................................................6 Solid state storage technology ...............................................................................................................7 SSD longevity ...................................................................................................................................8 Over provisioning .........................................................................................................................8 Wear leveling ..............................................................................................................................9 Wear Gauge ...............................................................................................................................9 SSD performance .............................................................................................................................9 IO Accelerators ..............................................................................................................................10 HP quality focus .................................................................................................................................11 Conclusion ........................................................................................................................................11 For more information ..........................................................................................................................12

Introduction
HP ProLiant Gen8 servers embody more than 150 design innovations. By engineering ProLiant Gen8 servers for enhanced solid state performance and by adding intelligent algorithms and analytics to manage performance, resiliency, and data protection in real-time, we made ProLiant Gen8 servers an all-in-one platform optimized for your storage workload demands. The direct attached storage (DAS) solution you choose for your systems affects their performance, cost, and usability. To find the best solution for your environment, you should take into consideration interconnect technology, controller technology and features, as well as drive types and configurations. This technology brief discusses all these considerations for direct attached storage to HP ProLiant Gen8 servers. Note that we include multiple types of DAS: Storage that connects directly to the drive controller or through an expander to the drive controller Storage within the server chassis or in an external chassis

Interconnect technology
Technologies to directly attach one or more drives to a computer system include Serial ATA (SATA) and Serial Attached SCSI (SAS). SATA and SAS have replaced parallel SCSI because of their higher throughput, simpler cabling, and support for more drives per port. Each drive with a SATA or SAS interface has its own high-speed serialized communication channel to the controller or expander.

Serial ATA
SATA uses a point-to-point, half-duplex, serial connection and the ATA command set. This command set is simpler but provides less robust functionality than the SCSI interface used with SAS. The SATA interface now has performance enhancements such as native command queuing (NCQ) and speed increases up to 6 Gb/s. SATA is the best solution for price-sensitive, low I/O workload applications.

Serial Attached SCSI

SAS uses a point-to-point, full duplex, serial connection and the SCSI command set. This command set provides better performance than SATA. Also the SCSI error handling protocol, which is superior to SATA, increases SAS reliability. SAS devices can have dual ports for link redundancy and can access the full bandwidth of a SAS link. SAS is the best solution for highest performance, high I/O workload, and mission-critical applications.

Interoperability
SAS shares physical characteristics with SATA. SATA drives can connect to SAS controllers operating at the SATA link speed and through expanders using SATA Tunneling Protocol (STP). You can use SATA and SAS drives in a single enclosure to meet capacity, performance, and cost goals; but you cannot mix them in the same logical array. In addition, you cannot use SAS devices with SATA controllers.

Expanders
SAS expanders allow one SAS port to connect to multiple drives or other expanders. HP SAS Expander Cards and ProLiant Gen8 Expander Backplanes allow servers to connect more drives per port, use more than eight internal drives in a RAID array, and spread a RAID array across multiple drive cages. 2

SAS/SATA link speed specifications and performance

The true performance of a drive is determined by how quickly the data can be accessed. In the case of rotational media, that includes time for the requested bits to spin under the headwhich is governed by the latency of track positioning, the areal density of the media, and the RPM of the drive. However, the speed of the interface electronics is often much greater than the speed of mechanically locating data on spinning media. The 6 Gb/s interface has an effective maximum theoretical bandwidth of 600 MB/s. Current SAS disk drives with high areal density and spinning at 15,000 RPM are capable of a maximum sustained throughput of about 200 MB/s for sequential data read access. That is less than one-third the bandwidth of the 6 Gb/s link. Random read and write performance can be significantly lower. When used in configurations with single disk drives connected directly to the controller, the 6 Gb/s interface and 6 Gb/s drives provide little or no performance advantage over 3 Gb/s. In these cases, the performance limiter is the disk drive throughput, not the link speeds. The performance benefits of 6 Gb/s components become important when constructing SAS fabrics and larger drive arrays using SAS expanders that support the 6 Gb/s link. These configurations can combine the I/O load of multiple drives onto a single link and take advantage of the additional bandwidth as shown in Figure 1. The controller and drives negotiate the link rate based on the maximum common rate supported by all devices from controller to drive.

Figure 1: The 6 Gb/s interface increases throughput for multiple drives on a port.

The 6 Gb/s SATA and SAS bandwidth is also important when you consider the emerging category of solid state drives (SSDs). We expect new generations of SSDs to support 500 to 600 MB/s throughput and be able to consume the entire bandwidth of a single 6 Gb/s link.

Drive controller technology

The drive controller helps determine the capacity, performance, and features available in a server storage system. Controller choices include embedded chipsets, PCIe host bus adapter (HBA) cards, and HP Smart Array Controllers. Our drive controllers are available with a variety of internal and external ports and features such as write cache and RAID arrays. 3

Host bus adapters

HP has embedded and PCIe interface HBAs available for environments that do not require the added security of hardware RAID but might need boot drives, JBOD, or software RAID arrays. HP H-Series HBAs support 6 Gb/s SATA and SAS hard disk drives (HDDs) and SSDs, backplanes within the server chassis, and SAS connection to external storage devices. These adapters also support multiLUN tape drives.

Smart Array controllers

HP Smart Array Controllers are powerful I/O solutions for the most demanding storage requirements. Smart Array Controllers allow adding drives for capacity growth. They have higher performance using more drives and additional RAID levels, better data availability, and advanced data security and management features. Our latest controllers have technology that increases performance and reliability: Roughly 3x IOPS (input/output operations per second) and 4x RAID 5 performance of our previous controllers, dependent on configuration Support for up to 200 drives with Smart Array Controller P421 Predictive spare activation by which the array controller rebuilds a pre-failure drive to a spare, before the drive fails Flash Backed Write Cache (FBWC) instead of battery backed write cache for reduced maintenance downtime and improved data security Advanced Data Mirroring (ADM) using 3 drives (enabled by SAAP 2.0)more reliable for data protection than traditional 2 drive mirror or 3 drive RAID 5 HP P-Series Smart Array Controllers offer advanced storage performance, availability, and security. Table 1 lists some characteristics of P-series controllers for our ProLiant Gen8 servers.

Table 1: Characteristics of HP Gen8 P-Series Smart Array storage controllers P220i Blade/Rack PCI Bus Memory Options Blade Embedded PCIe 2.0 x8 512 MB 40 bit FBWC P222 Rack Low profile PCIe 3.0 x8 512 MB 40 bit FBWC P420 Rack Low profile PCIe 3.0 x8 512 MB 40 bit, 1 GB and 2 GB 72 bit FBWC 2 x4 ports miniSAS internal P420i Rack Embedded Zero memory P421 Rack Low profile PCIe 3.0 x8 1 GB or 2 GB 72 bit FBWC

SAS/SATA Connectivity

1 x4 port mini-SAS internal Up to 4

1 x4 port internal mini SAS and 1 x4 external Up to 4 internal, up to 100 external 0, 1, 1+0, 5 and 50

2 x4 ports miniSAS internal

2 x4 ports mini-SAS external Up to 200 external 0, 1, 1+0, 5 and 50

Max Drives*

Up to 27 internal 0, 1, 1+0, 5 and 50

Up to 27 internal 0, 1, and 1+0, 5 and 50 with optional memory

RAID Support

0 and 1 for Blades

Smart Array Advanced Pack (SAAP) 2.0 Support

optional 6, 60, and ADM with software key

optional 6, 60, and ADM with software key

optional 6, 60, and ADM with software key

optional 6, 60, and ADM with software key

*Depending on available bays

Storage infrastructure
For ProLiant Gen8 servers and BladeSystem servers our technology for packaging drives and connecting them internally and externally has evolved for higher densities, improved manageability, and simplified cabling.

HP SmartDrive carriers
The new SmartDrive carriers for ProLiant servers have fault and activity indicators and a microcontroller to monitor and store information about drive operation and status. We reduced the drive carrier dimensions to allow up to a 50% increase in server drive count. SmartDrive Authentication technology verifies that you are using genuine HP drives. HP SmartDrive carriers do not fit in ProLiant platforms prior to Gen8 servers. Likewise, drives designed for G7 and earlier systems will not fit into our Gen8 servers and server blades.

Internal expander backplanes

HP developed new expander backplanes for use in ProLiant Gen8 server platforms. Unlike previous designs, these backplanes do not require a SAS expander in a PCIe slot. They have simplified cabling and support SmartDrive status monitoring and error logging. 5

BladeSystem direct attached storage

HP BladeSystem servers can support two types of direct attached storage: drives on the blade servers and drives in external storage enclosures. HP SAS storage for BladeSystem servers support zoned direct attached storage as well as shared SAS storage to blade servers. This SAS architecture consists of an HP Smart Array Controller and a SAS switch connected to an HP Modular Disk System (MDS) or Modular Smart Array (MSA) storage enclosure. The Virtual SAS Manager interface, accessed from the SAS switch, allows users to dynamically zone drive bays to specific blade servers. HP Smart Array Controllers can access SAS or SATA drives installed in the zoned drive bays as local and dedicated storage.

HP hard disk drive categories

At HP, we have refined and expanded our HDD family into three distinct categories: Entry, Midline, and Enterprise (Table 2). These categories meet the needs of different environments for performance, reliability, and cost-capacity. HP Enterprise drives give you maximum reliability, highest performance, scalability, and error management under the most demanding conditions. They are the only HP drives designed for unconstrained I/O workloads and for mission-critical environments such as large databases, email servers, back-office applications and virtualization. HP Midline drives are for high-capacity applications that require increased performance and reliability. Midline drives are more resistant to rotational and operational vibration than HP Entry drives and so are better suited for multi-drive configurations. Midline drives are available with either SATA or SAS interfaces and are for use in moderate workload environments. We do not recommend Midline or Entry drives for mission-critical applications. HP Entry drives have the lowest unit cost and give you a basic level of reliability and performance. They are best suited for non-mission-critical environments where I/O workloads are 40% or less. They are typically best suited for internal or archival storage, or as boot drives for entry-level servers. Entry drives are only available with a SATA interface.

Table 2: Characteristics of HP hard disk drives for servers Entry drives* General description Use environments Lowest unit cost Boot drive Non-critical storage Midline drives High capacity Lowest cost per gigabyte External storage Backups/archival Redundancy Workload Reliability Interface Connectivity RPM Warranty SATA 3 Gb/s Single port 7,200 1 year < 40% < 40% 2 X Entry drive reliability SATA 3 Gb/s and 6 Gb/s SAS 6 Gb/s Single port SATA Dual port SAS 7,200 1 year 10K and 15K 3 years Dual port Unconstrained workloads 3.5 X Entry drive reliability SAS 6 Gb/s Enterprise drives High reliability High performance Mission critical High I/O

*For comparison only. Entry drives are not currently available as options for ProLiant Gen8 servers.

HP drives for servers are available in both 2.5-inch (small) and 3.5-inch (large) form factors. SFF drives can require as little as half the power and generate significantly less heat than LFF drives. SFF drives are now the primary storage form factor because of their increased data densities, better reliability, lower power requirements, and smaller size. Our new Enterprise class drives and all new 10K and 15K RPM drives are SFF, and SFF drives are standard in our Gen8 servers. To use LFF drives in Gen8 servers, you must order specific server configurations. The industry is transitioning away from LFF enterprise drives, though they are the better choice when you need large capacities and lowest cost per gigabyte. We expect to continue developing LFF Midline class drives.

Solid state storage technology

Solid state storage delivers higher performance, lower latency, and lower power consumption than traditional rotating media. Not all solid state storage is the same, so the HP portfolio includes three classes of solid state drives: Enterprise Performance SSD, Enterprise Mainstream SSD, and Enterprise Value SSD. These device classes meet the requirements of different applications based on capacity, performance, cost, and endurance (Table 3).

Table 3: Characteristics of HP solid state drive categories Enterprise Value Interface(s) General description Capacities NAND technology Workload Reliability/ endurance Data retention* Usage environment 3 Gb/s SATA 6 Gb/s SATA SFF and LFF hot plug 60 480 GB MLC High read/low write 3 year service life with highly constrained write workloads 3 months minimum Boot devices, applications high in reads, few or no writes, or data is transient Enterprise Mainstream 3 Gb/s SATA 6 Gb/s SAS SFF and LFF hot plug 200/400/800 GB SLC and MLC Balanced read/write workload 3 year service life with constrained write workloads < 3 months High IO/s applications with equal read/write workloads SFF hot plug 200/400 GB SLC High read/write workloads 3 to 5 year service life with unconstrained workloads < 3 months Mission-critical/unrestricted workload, high IO/s applications Enterprise Performance 6 Gb/s SAS

* With no power and drive near end of life

SSD longevity
Generally speaking, we expect electronics with no moving parts such as an SSD to have longer lifetimes than mechanical devices such as a HDD, which has moving parts that can fail. The lifespan of NAND memory in SSDs depends on the type of cell structure and the number of write/erase cycles it experiences. Multi-level cell (MLC) memory typically has less endurance than single-level cell (SLC) memory. SLC is less dense and therefore more expensive than MLC for the same capacity. We use both cell types, depending on capacity and endurance goals. HP SSDs include technologies to increase endurance to meet the requirements of enterprise environments. Several factors affect the endurance of SSDs: SSDs write NAND pages, not sectors. SSD writes generate a read/modify/write cycle. SSDs must erase a NAND block (~1 MB) before writing a page (~4 KB) in that block. For write performance, SSDs maintain a pool of erased blocks. SSDs can move data from sparsely populated blocks so those blocks can be erased (known as garbage collection).

Over provisioning
HP drives contain extra blocks, as many as 25% more blocks than the drives stated capacity. SSD drives use this extra area to distribute erases and writes across a larger number of blocks over time. Erasing blocks increases write performance and lowers write amplification. Write amplification is the ratio of the number of NAND writes needed to accomplish host writes. A lower write amplification increases SSD longevity. 8

Wear leveling
HP wear leveling technology uses sophisticated algorithms to re-map logical blocks receiving frequent writes to different physical pages. It evenly distributes erasures and rewrites across the storage medium to maximize endurance.

Wear Gauge
HP ProLiant and BladeSystem servers have HP SMARTSSD Wear Gauge technology. Through configuration and management software, it monitors and reports the percentage of SSD drive life used and the amount of life remaining under the workload-to-date. Alerts notify you when little drive life remains so you can replace a drive before it fails.

SSD performance
Enterprise Value, Mainstream, and Performance SSDs all deliver I/O performance comparable to Enterprise HDDs. SSDs differ from HDDs primarily by the read/write workload levels they can support (Table 4) and their expected service life: Enterprise Value SSDs provide relatively large storage capacities at lower costs, but they do not have the endurance of the Mainstream or Performance SSDs. Enterprise Mainstream SSDs have smaller capacities but greater endurance than Value SSDs. Enterprise Performance SSDs have capacities similar to Mainstream SSDs but have greater endurance. The higher data rates of SSDs can increase the probability that an unreported data bit error can occur. Our Enterprise class SSDs and HDDs have full data-path error detection. Some of our drives also have error correction. Our SSDs have power loss protection. It ensures that if a drive loses power, including hot plug removal, it can recover quickly when power is restored. Our SSDs fit seamlessly into the existing HP server infrastructure and deliver exceptional performance in high random-read IOPS applications as shown in Table 4. All performance values are typical and for comparison only.

Table 4: Performance comparison of a SAS HDD with SSDs for servers SFF SAS disk drives (15K RPM) Technology 6 Gb/s SAS Value SSD 3 Gb/s or 6 Gb/s SATA MLC NAND Sequential writes (64 KB) Random writes (4 KB, 16 commands outstanding) Sequential reads (64 KB) Random reads (4 KB, 16 commands outstanding) 380 IOPS 30,000 IOPS 30,000 IOPS 40,000 IOPS 160 MB/s 200 MB/s 230 MB/s 400 MB/s 300 IOPS 300 IOPS 160 MB/s 100 MB/s Mainstream SSD 3 Gb/s SATA MLC/SLC NAND 6 Gb/s SAS MLC NAND SATA MLC 225 MB/s SAS MLC 120 MB/s 10,000 IOPS 15,000 IOPS 300 + MB/s Performance SSD 6 Gb/s SAS SLC NAND Dual port

IO Accelerators
HP IO Accelerators are a direct attach, solid state storage system in PCIe card format for ProLiant servers and in Mezzanine format for BladeSystem servers. I/O accelerators use SLC or MLC NAND flash memory technology. These devices are ideal for applications requiring high transaction rates and real-time data access such as database and database acceleration, Web servers, video, rendering, and animation. I/O accelerators allow applications to access them like other storage volumes. But an I/O accelerator is both a controller and a storage device, with its own specialized driver that translates standard block level I/O into NAND reads and writes. It delivers data directly across the PCIe bus, resulting in lower latency in specific environments. The I/O accelerator architecture leverages the greater bandwidth and multi-core processing capabilities of server CPUs. An I/O accelerator can therefore achieve higher block storage throughput and lower latencies than traditional storage. Error correction and wear-out monitoring and prediction maintain data integrity, but RAID fault tolerance is not available in IO Accelerator solutions. Table 5 lists some performance data for our IO Accelerators.

10

Table 5: Performance data for HP IO Accelerators 160 GB ioDrive Write bandwidth Read bandwidth IOPS (75% reads) Read access latency 750 MB/s 770 MB/s 123,000 26 s 320 GB ioDrive 510 MB/s 735 MB/s 67,000 29 s 320 GB ioDrive Duo 1.5 GB/s 1.5 GB/s 238,000 26 s 640 GB ioDrive Duo 1.0 GB/s 1.5 GB/s 138,000 29 s 1.28 TB ioDrive Duo 1.1 GB/s 1.5 GB/s 150,000 30 s

HP quality focus
Our high standards ensure that all of our drives are reliable and integrate smoothly into HP server and storage systems. We refine our processes continuously to ensure ongoing improvement in both current and future products. We have an extensive qualification program where HP engineers work closely with drive suppliers to determine testing procedures and metrics that a drive must meet. The test population typically requires approximately 1,000 individual drives to evaluate a product family during the selection-evaluation and development-verification steps. These two steps require approximately two million drive-hours of testing. Drive products that pass the HP qualification process move into the HP drive performance-monitoring phase during volume production, where continuous improvement is possible. HP engineers and the drive suppliers work as a team during the volume production phase of a product. The team monitors each products performance through quality control methods at the suppliers factory and at HP option kitting configuration sites. We review product quality data on a daily, weekly, and monthly basis.

Conclusion
Because of the advancements in storage technology, you can choose from a broad spectrum of direct attach storage solutions, each with unique capacity, performance, and reliability characteristics. When considering which technology to choose for your storage needs, consider all factors that contribute to the solution. The interface type affects performance, future expansion, and reliability of the storage system. Serial Attached SCSI has advantages in these areas over the SATA interface. Controller technology determines system performance, capacity limits, manageability, and data security. For all but smaller, lower performance systems, choosing an appropriate Smart Array Controller adds state-of-the-art availability and security options to your system. Drives perform best when matched to the goals of the storage system. Our portfolio of hard disk drives satisfies capacity, performance, and cost goals. Add solid-state storage for the highest performance and data availability when endurance limits and cost are justified, and IO Accelerators where lowest access latency is required. HP has industry-leading storage technologies to handle all of your requirements. Our storage products are rigorously tested and certified for their targeted applications.

11

For more information

Visit the URLs listed below if you need additional information.
Resource description ProLiant Storage white papers HP Storage for ProLiant Servers HP on Solid State Storage Technology Web address www.hp.com/servers/technology

http://h18004.www1.hp.com/products/servers/platfor ms/storage.html
http://www.hp.com/go/solidstate

Send comments about this paper to TechCom@HP.com Follow us on Twitter: http://twitter.com/ISSGeekatHP

Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. TC1108769, March 2012