Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Table of contents
Executive summary ...................................................................................................................................................................... 2
Overview .......................................................................................................................................................................................... 2
Solution components ................................................................................................................................................................... 3
HP 3PAR StoreServ Storage ................................................................................................................................................... 3
HP ProLiant Servers.................................................................................................................................................................. 8
Microsoft Windows Server and SQL Server ......................................................................................................................... 8
VMware vSphere 5.1 ................................................................................................................................................................ 9
Test environment ........................................................................................................................................................................ 10
3PAR StoreServ Storage........................................................................................................................................................ 11
HP ProLiant Servers................................................................................................................................................................ 21
MS SQL Server 2008 R2 and MS SQL Server 2012 guidance ........................................................................................ 22
VMware vSphere for Failover Clustering ............................................................................................................................ 24
Key findings and test scenarios ............................................................................................................................................... 25
SQL Server workloads used .................................................................................................................................................. 25
Single VM failure with SQL Server ....................................................................................................................................... 25
HP 3PAR Dynamic Optimization .......................................................................................................................................... 26
HP 3PAR Adaptive Optimization .......................................................................................................................................... 26
HP 3PAR Priority Optimization (QoS) .................................................................................................................................. 29
Summary ....................................................................................................................................................................................... 32
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Executive summary
HP 3PAR StoreServ Storage provides superior performance and availability from mid-range to high-end storage through
industry leading architectural innovation, design, and implementation. Specifically, its tightly coupled clustered architecture,
secure multi-tenancy, and mixed workload support are designed to deliver enterprise IT as a utility service simply,
efficiently, and flexibly. It is the only storage platform spanning the low end of the midrange to the high end of the
enterprise market segments with a single architecture. HP 3PAR StoreServ is an excellent choice for both individual
SQL Server environments and consolidated SQL Server environments with VMware vSphere.
HP 3PAR StoreServ offers a robust storage solution that builds redundancy, performance, and breakthrough functionality
on top of common components, driving a huge return on investment, in a virtualized environment. Some of the
breakthrough features of HP 3PAR StoreServ include fine-grain, wide striping of data on all available storage for
exceptional performance, quality of service (QoS) capabilities, VMware vSphere Storage API integration, vCenter plug-ins,
HP 3PAR thin technologies, and storage tiering.
When servers are consolidated on virtual machines (VMs), they tend to squeeze storage performance as the array becomes
the focal point of a greater I/O demand than ever before. Workloads in virtualized environments are unpredictable, with
random and heavy I/O patterns. As a result, storage must be resilient enough to handle numerous hosts and mixed
workloads. HP 3PAR StoreServ provides performance and tuning capabilities to meet demanding workload needs.
VMs are only as available as the storage on which they reside. Storage in virtualized environments must be protected at
many levels, including disks, controllers, and nodes, to enable survival without disruptions should failures occur. With
redundant controllers, HA drive and HA cage configurations, online servicing, Persistent Cache, snapshot and copy
functionality, and online updates, HP 3PAR StoreServ can meet the highest levels of uptime. Additionally, HP 3PAR
StoreServ has native phone home capabilities to alert HP support in the event of a failed component, so that the part
can be quickly replaced without degrading performance or creating large degraded availability windows.
Virtualization introduces complexity into the data center, directly affecting the underlying storage infrastructure. Storage
must be as simple to manage and maintain as possible and not add unnecessary complexity to the virtualized environment.
HP 3PAR StoreServ is designed for virtualization and delivers easily managed, consolidated storage for virtual environments
with the increased resiliency and performance unavailable in smaller, stranded storage environments.
Storage capital expenditures often detract from virtualization implementation savings. Storage should be enhanced for
virtualized environments to maximize efficiency, so capacity and performance barriers do not create additional expenses.
HP 3PAR StoreServ consolidates thickly provisioned, older existing environments and recaptures stranded storage, while
migrating data into thinly-provisioned, wide-striped storage (called HP 3PAR Thin Conversion). HP 3PAR StoreServ takes
advantage of adaptive capabilities to help resolve persistent hot spots. Deploy HP 3PAR StoreServ for the storage
performance needs of today, without overprovisioning for unknown needs next year.
Target audience
This white paper is intended for IT decision makers, database administrators, storage administrators, system administrators,
and database and solution architects involved in the planning and deployment of consolidated MS SQL Server environments
with VMware vSphere 5.1 on the HP 3PAR StoreServ platform.
Overview
HP 3PAR StoreServ is optimized for vSphere and SQL Server
Consolidating workloads in virtual server environments is key to driving increased savings. HP 3PAR StoreServ offers wide
striping, mixed workload support, and integration with VMware Adaptive Queue Depth Throttling algorithm and vSphere
Storage APIs, providing the performance necessary to increase VM density for each server by two foldguaranteedwhen
compared to VMware deployments using traditional storage arrays. 1
1
HP 3PAR Guarantee!
h20195.www2.hp.com/V2/GetDocument.aspx?docname=4AA4-0948ENW&cc=us&lc=en
www8.hp.com/us/en/products/data-storage/data-storage-solutions.html?compURI=1284392
media.hp.com/program.aspx?key=NOMMKJMJMF
2
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Because a VMware vSphere server can host many VMs, each with its own I/O patterns, the resulting I/O pattern seen by the
array is generally random in nature. Random I/O means that the server derives less benefit from storage caching than a
server with a more predictable access pattern, forcing more requests to access disks. Thus, the number of drives that make
up the LUN predominantly determines performance. Wide striping spreads even modestly-sized volumes across as many
drives as possible in HP 3PAR StoreServ. This results in unmatched I/O performance, even when caching is minimally
effective. This specifically benefits SQL Server environments, where even the smallest databases benefit from the
performance of all the available disks in the array.
Virtualizing Microsoft SQL Server workloads with VMware vSphere on HP 3PAR StoreServ is a way to achieve performance
and increase hardware resource usage throughout your data center. HP 3PAR StoreServ provides sustained performance
for SQL Server and other consolidated workloads. When SQL Server storage capacity or performance needs to increase,
HP 3PAR StoreServ online storage upgrades meet the demand. The HP 3PAR StoreServ accommodates increasing SQL
Server OLTP performance demands by adding additional disk storage, up to the maximum number of configurable spindles.
HP 3PAR StoreServ, VMware virtualization, and Microsoft SQL Server provide functionality, performance, and ease of use,
helping todays businesses lower costs, reduce business risk, and improve IT agility.
Solution components
Components tested for this paper included HP 3PAR StoreServ Storage, HP ProLiant Blade Servers, VMware
vSphere 5.1, and Microsoft SQL Server in consolidated, clustered environments on Microsoft Windows 2008 R2,
and Windows Server 2012.
2
HP 3PAR StoreServ Storage: hp.com/go/3PAR
3
HP 3PAR StoreServ 7000 Storage QuickSpecs: h18000.www1.hp.com/products/quickspecs/14433_div/14433_div.pdf
4
HP 3PAR StoreServ offering: www8.hp.com/us/en/products/disk-storage/index.html?facet=3par-storage
3
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Databases are considered a mission-critical application for many customers. Careful performance planning allows for the
creation of a QoS policy that helps ensure the proper day-to-day operation of a database, while addressing runaway queries
that may consume excessive IOPS and bandwidth.
Priority Optimization is the only quality of service solution supported for a virtualized MS Cluster Services environment
running on VMware VMs across hosts with shared RDM clustered storage. VMware does not support Storage I/O Control
(SIOC) with Raw Device Mapping (RDM), as noted in VMware KB 1022091. SIOC throttling is based on I/O latency; Priority
Optimization is based on max IOPS and bandwidth.
Priority Optimization adds additional I/O tuning capabilities to HP 3PAR StoreServ by enhancing the heterogeneous I/O
handling capabilities already present. Priority Optimization operates by setting maximum thresholds for front-end IOPS
and bandwidth at the Virtual Volume set (VVset) level.
Priority Optimization key features provide these benefits:
Simplifies implementation, monitoring, and troubleshooting.
Complements Adaptive Optimization.
Requires no physical pre-planning or training.
Works for every tier of Storage, not only SSD.
Works for both Bandwidth and IOPS.
Works the same on all platforms, from the 7200 to 10800 (including T and F class).
Requires no Host Agents.
When an I/O packet reaches the HP 3PAR StoreServ controllers, HP 3PAR Priority Optimization takes one of these actions:
Passes the I/O packet to the Virtual Volume
Delays the I/O by stalling it in a private QoS queue
Returns a SCSI Queue Full (QFULL) message to the host
If the upper limit for IOPS or bandwidth for a particular VVset is reached, HP 3PAR Priority Optimization delays SCSI I/O
request responses to the host for the volumes contained in that VVset. Every QoS rule maintains its own queue for delayed
I/Os. Only the I/O request descriptions are queued, not the actual data.
5
HP 3PAR Priority Optimization white paper: h20195.www2.hp.com/V2/GetPDF.aspx/4AA4-7604ENW.pdf
4
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
A controller nodes cache is not impacted because the QoS rules are applied before write I/O data hits the cache. When I/O
requests reside longer than 200 ms in a QoS queue, then addition incoming I/Os to the volumes in the VVset are rejected,
and a Queue Full (QFULL) response is returned to the host server using the volumes. QFULL prevents delayed I/O from
holding all system resources, such as host, HBA, and VV layer buffers and queues. When configured correctly, vSphere will
respond to the QFULL message appropriately and throttle I/O. The I/O delay and the eventual QFULL response apply to all
members of the VVset, even if only one of the VVs causes the QoS threshold breach.
When setting QoS rules, consider not just the current performance workloads, but also how those workloads change over
time. Some variables that affect QoS are:
r/w ratios
block size
sequential vs. random access patterns
cache hits
Priority Optimization test results are shown in the Test scenarios section of this paper. Priority Optimization is a feature
of HP 3PAR OS 3.1.2 MU2 and greater, and is supported on all HP 3PAR StoreServ Storage systems that are certified for
this version of 3PAR OS. This includes the 3PAR InServ F-Class and T-Class, as well as the 3PAR StoreServ 7000 and
10000 systems.
HP 3PAR Thin Provisioning 6
HP 3PAR StoreServ provides value-added thin functionality to increase return on investment and simplify management
and maintenance.
HP 3PAR Thin Provisioning eliminates the stranded storage dilemma plaguing many storage implementations and is used
for storage needs including RDM devices, because this enables the most cost-effective storage while reducing
administrative effort. VMware vSphere support for Microsoft clustering solutions requires raw device mapping (RDM)
for shared storage (VMware KB 1037959 7). When implementing HP 3PAR Thinly Provisioned Virtual Volumes (TPVVs),
administrators often ask whether additionally implementing vSphere Thin Provisioning for VMDK files is beneficial.
In general, both Thin Provisioning with 3PAR and vSphere accomplish the same end-result, albeit at different logical layers.
With VMware vSphere Thin Provisioning, administrators realize greater VM density at the virtual machine file system (VMFS)
layer at the cost of CPU and disk I/O overhead, as the volume is incrementally grown on the ESXi hosts. By implementing
HP 3PAR TPVVs, the same VM density levels are achieved; however, the thin provisioning CPU work is offloaded to the
HP 3PAR ASIC.
If the goal is to reduce storage costs, maximize storage usage, and maintain performance, then use HP 3PAR Thin
Provisioning Software to provision virtual volumes. If performance is not a concern, but overprovisioning VMs at the
VMFS layer is important, then implement both thin provisioning solutions. However, there are no additional storage
savings realized by using vSphere Thin Provisioning on top of HP 3PAR TPVVs. In fact, implementing both solutions adds
more management complexity to the environment.
Note:
HP recommends using only HP 3PAR Thin Provisioning, which reduces management complexity and enables greater
performance and efficiency.
6
HP 3PAR Thin Technologies, www8.hp.com/h20195/v2/GetDocument.aspx?docname=4AA3-8987ENW
7
VMware KB 1037959, Microsoft Clustering on VMware vSphere: Guidelines for Supported Configurations.
5
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Note:
Reclaiming space when snapshots are deleted with the Thin Copy Reclamation feature requires the Virtual Copy,
Remote Copy, or Thin Provisioning license.
8
HP 3PAR Adaptive Optimization Software, www8.hp.com/us/en/products/storage-software/product-detail.html?oid=5043778
6
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
AO is relatively easy to schedule and run. Use the intuitive user interface, shown in figure 2, to perform the following tasks:
Start and stop AO immediately or at a scheduled time.
Set a start time for the monitoring period, which precedes data migration and can run for up to seven days.
Optionally, set a stop time to end the monitoring period at any time before the actual migration. Monitoring during a
period of heavy activity, but migrating later when storage I/O is reduced, helps to minimize the effects of data migration
on critical work. Notice that the monitoring period can actually involve time in the past.
Specify a data migration period from 1 to 24 hours.
Execute a dry run to monitor and analyze data, without actually performing a migration.
Data/business protection
HP 3PAR StoreServ protects data with a flexible and efficient copy-on-write technology that maximizes both recovery point
objectives and recovery time objectives.
HP Storage Services 9
HP provides professional support of your HP 3PAR StoreServ environment, including proactive advice, performance
baselines, and deep performance analysis when you do not have the expertise in-house, or need an expert set of eyes to
ensure sound configuration decisions.
9
HP Storage Technology Services: www8.hp.com/us/en/business-services/it-services.html?compURI=1078604
7
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
HP ProLiant Servers
The servers used consisted of the industrys best-selling HP ProLiant BL460c Gen8 blade servers and
HP ProLiant BL460c Gen7 blade servers. 10 Designed for virtualization from the ground up, HP ProLiant Server blades
deliver more memory and superior networking capability for virtualized environments and compute-intensive
application workloads.
The HP c7000 blade enclosure provided the network and infrastructure for the blade servers. HP Virtual Connect FlexFabric
10Gb/24-port Modules are the simplest, most flexible way to connect virtualized server blades to data or storage networks.
VC FlexFabric modules eliminate up to 95% of network sprawl at the server edge, with one device that converges traffic
inside enclosures and directly connects to external LANs and SANs. 11
10
HP BladeSystems: hp.com/go/bladesystem
11
HP Virtual Connect FlexFabric 10/24: h30094.www3.hp.com/product/sku/10276336
12
MSDN AlwaysOn FCIs: msdn.microsoft.com/en-us/library/ms189134.aspx
13
MS SQL Server, Mission Critical Confidence: microsoft.com/en-us/sqlserver/solutions-technologies/mission-critical-operations.aspx
14
MS SQL Server features in base product: microsoft.com/philippines/server/sqlserver/compare-competitor.aspx
8
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Running SQL Server 2012, SQL Server 2008 R2, and other consolidated solutions on VMware vSphere 5 VMs with HP servers
and storage offers the benefits of server consolidation, operational efficiency and high availability, configuration flexibility,
and the use of HP Insight Control integration with VMware vCenter to simplify management.
Converting a physical server to a VM is a convenient method of replacing aging hardware. Converting multiple servers to
VMs and consolidating them to fewer physical hosts creates significant savings in equipment, power, cooling, and real
estate. When sizing the host servers, confirm that they have at least the sum of the used resources on the physical servers
being converted to VMs. Additionally, provide resources to handle the hypervisor overhead.
For access to tools that can help in your sizing efforts, see the section Resources.
15
For Microsoft Cluster Service (MSCS) support on ESXi/ESX (1004617) refer to kb.vmware.com/kb/1004617
9
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Figure 4 shows a portion of the information displayed in the VMs To Volumes information screenshot. Columns of
information can be hidden or displayed to suite specific needs. Figure 4 shows a portion of the useful information collected
by the HP Insight Control module, and displayed in one place. A horizontal scrollbar is used to display several pages of VM to
Volume end-to-end information.
Figure 4. HP Insight Control Storage Module showing a portion of VMs To Volumes information display
Test environment
Figure 4 shows the hardware layout, including the HP c7000 Blade enclosure front view and the VMware virtual machines
that host the MS Windows Failover Clustering cluster and SQL Server instances. HP c7000 VC FlexFabric 10Gb/24-port
modules provide the network resources used for vSphere vSwitches and dvSwitches. Four HP B-Series 8/24c SAN Switches
provide storage connectivity.
16
HP Insight Control Storage Module documentation: h18004.www1.hp.com/products/servers/management/unified/infolibraryicv.html
HP Insight Control Storage Module download: h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=HPVPR
10
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
11
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Item Description
Number of drive chassis and type5 total 1x 2U controller with 24 SFF slots
3x 2U 24 SFF slots
1x 4U 24 LFF slots
HP 3PAR Dynamic Optimization license Full volume (or LUN) migration license
HP 3PAR Adaptive Optimization license Sub volume (or LUN) region migration license
HP 3PAR Provisioning
This paper is not a sizing guide, nor does it suggest the maximum capabilities of the equipment used.
One benefit of virtual environments is they are generally more forgiving of sizing issues. With virtualization, VMs can be
created, copied, migrated, and modified. Likewise, the HP 3PAR StoreServ, with thin provisioning technology built in,
can present over provisioned LUNs that are completely redundant and simplify administration. This environment used
3PAR Thinly Provisioned Virtual Volumes (TPVVs) for SQL Server data and log LUNs to simplify administration. Thin
provisioned volumes provide maximum flexibility, but if capacity usage will not be growing, then thick provisioned volumes
will be adequate. Thin VLUNs can be over provisioned at the Windows OS level, so precise sizing of the LUN is not needed.
LUN provisioning with HP 3PAR StoreServ is relatively straightforward for those familiar with vSphere. Use either the
HP 3PAR Management Console or the HP Insight Control Storage Module to create vSphere datastores. Use the HP 3PAR
Management Console to create the needed RDMs.
Table 2 shows initial LUN configurations, which were predominately thin provisioned. Volumes used were of type RAID 1 and
RAID 5, with RAID 5 being preferred for maximum flexibility and value. RAID 5 was selected for the majority of use cases.
The HP 3PAR StoreServ controllers will buffer the write I/Os in cache, providing quick response times as long as the back
end disks are able to keep up with the asynchronous cache flushes. If back end disk I/O becomes too heavy for the disks to
handle, then several options are available.
If additional storage capacity was planned, then add disks to the CPG. With HP 3PAR StoreServ wide striping of data
across all disks in a CPG, additional disks should provide additional backend performance.
Consider using HP 3PAR Priority Optimization to limit host I/O and throughput of lower tier I/O workloads, so array
performance is maintained for critical tier 1 workloads.
Create a RAID 1 CPG and move VVs with the highest write workloads to the RAID 1 CPG.
If SSDs are available, and the complete VV with the highest tier 1 I/O demand can reside on SSD storage, then create an
SSD CPG and move the VV to SSD storage.
12
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
If the regions of highest I/O workload are intense enough, and dense enough (very high I/O in a small amount of space),
and the I/O hot spots do not deviate drastically over time, then use 3PAR Adaptive Optimization to find and move those
intense I/O regions to SSDs.
Use a combination of the options.
LUN Usage Primary CPG 17 Initial RAID Thick/Thin Size (GB) Comment
Note:
Consider references to FC HDDs or Performance SAS HDDs to be synonymous.
Moving a VV from a RAID 5 CPG to a RAID 1 CPG involves changing or tuning a Virtual Volumes properties. Tunable Virtual
Volume characteristics include RAID level, CPG, and thick/thin characteristic. Figure 6 shows how to easily tune a
datastore LUN containing vmdk boot files from RAID 5 to a RAID 1.
17
Primary CPG is the one from which new data is allocation from. This is important for Thinly Provisioned Virtual Volumes from CPGs that
are members of an Adaptive Optimization configuration.
13
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Selecting storage media can be difficult. The following use cases can help you decide which storage media to use for
SQL Server environments.
1. Capacity is important, while performance is not. This is likely not a SQL Server use case.
2. Capacity and performance are both important.
3. Capacity is not important, while performance is important.
For the first use case, in which performance is not important, use NL devices. This is likely not a good solution for any kind of
SQL Server consolidation environment.
In the second use case, both capacity and performance are important. This will likely be the need for the majority of
consolidation environments. Always size for IOPs first. To meet SQL Server performance requirements, spindle count is
important. Use large-capacity Fast Class storage to meet performance requirements. If additional capacity is needed, add
small NL drives, but no more than 25% of the total capacity. Adaptive Optimization can help place little-used data onto NL
devices, but data accessed from NL devices will have a performance penalty. If data is accessed evenly over the full capacity,
then do not use NL devices.
14
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Consider using several AO configurations, depending on the SQL Server needs. For tier 2 and 3 database needs, NL and FC
tiered storage configurations using AO will work for data with good localized access patterns that change little over time.
For tier 1 database needs, a 2 (or possibly 3) tier AO solution may work well.
In the third use case, performance is paramount. For SQL Server consolidation environments, this is the most difficult
environment to size when cost is also a concern. In this case, capacity is not important, but performance is, so do not use NL
devices. For very small capacity, high performance needs, an SSD-only solution is an excellent choice. SSD solutions provide
extremely high performance with minimal power and cooling, which is important for some use cases.
When HDDs are used with SSDs and AO, follow the AO considerations and guidelines discussed previously and in the
HP 3PAR Adaptive Optimization section of this paper. The HP 3PAR StoreServ 7000 has controllers that can run with a set
of 8 SSDs and still have performance head room, so running a 2 tier AO solution is a good option for this third use case if the
I/O usage profile is truly hot enough, localized enough, and changes little over time.
If possible, start with a smaller configuration and analyze data access patterns before implementing a tiered solution with
AO. This allows sizing of the tiers to match the solution needs. A safe starting point is to size the storage solution initially
with Fast Class HDDs and then add tiers based on actual use case data.
Use Priority Optimization to limit the potential performance impact a Tier 2 or Tier 3 SQL Server database might have on a
Tier 1 database environment. This also applies to other lower priority workloads, such as non-production environments,
and test environments that might be using the shared storage. When setting QoS rules, start conservatively and set
limits high.
Table 5 suggests when to apply the QoS configuration. Creating a separate volume set for each VV allows fine-tuning of QoS
configurations on an individual VV basis. Grouping multiple VVs into a single VVset provides a simple way to set QoS for all
those VVs with a single configuration set, but this might not be practical. QoS settings for a VVset apply globally, so one VV
might use more resources than anticipated and starve the other VV in the set. If you want to change the rule of an individual
VV in a set, either create a new VVset with that VV, or relocate the VV into its own VVset. Enabled rules for QoS VVsets are
applied using wired-OR logic, so the first QoS limit reached takes precedence.
15
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Priority Optimization
Usage Comment
Usage
Data (Tier 1
No Only to limit run-away processes or anomalous performance usage.
applications)
Data (Tier 2
Mixed Mild limits based on SLAs so Tier 1 performance is not compromised.
applications)
Data (Tier 3
Limit Aggressive limits based on SLAs so Tier 1 performance is not compromised.
applications)
When the special QoS VVset called System or sys: all_others, is used, all VVs not in another enabled QoS VVset will be
included in this catch-all category. HP recommends that a named QoS rule for a workload be enabled before starting the
workload if its I/O profile is known. This way, the workload will not be governed by the all_others rule whose IOPS and
bandwidth limit may be far too small. HP recommends that explicit QoS rules be defined with sensible limits for IOPS and
bandwidth on all VVsets used in Tier 1 applications. The all_others rule should always be defined to control I/O to
applications on the system without an explicit enabled QoS rule, to prevent them from affecting Tier 1 applications.
HP does not encourage the use of many QoS rules applied to the same VV. For this reason, a VV may be a member of only
up to eight QoS enabled VVsets. When a VV is a member of several QoS VVsets, the first VVset to reach a configured rule
limit is the rule that is applied to the VV. The maximum number of active QoS rules per an array is limited only by the
number of VVs times eight.
Table 6. Example usage of volume sets for Priority Optimization configurations
Gold App1,, Gold AppX Separate QoS VVsets for each Tier 1 application
Gold App1,, Gold AppX Separate QoS VVsets for each Tier 1 application
Silver App1,, Silver AppX Separate QoS VVsets for each Tier 2 application
Gold App1,, Gold AppX Separate QoS VVsets for each Tier 1 application
Silver App1,, Silver AppX Separate QoS VVsets for each Tier 2 application
Bronze App1,, Bronze AppX Separate QoS VVsets for each Tier 3 application
16
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Figure 7 shows a powerful level of control provided by QoS-nested rules. These nested rules functioned very well and
provide for overprovisioning of individual SQL Server databases or LUNs while controlling overall limits. As shown in the
figure, Tenant #1 is capped at 20k IOPs overall, while the three individual applications under Tenant #1 are capped at 10k
each. Individual applications 1, 2, and 3 can each peak to 10k IOPs, but not at the same time, since the Tenant #1 rule would
limit the I/O to 20k overall. The total Front-End IOPs are limited to 70k.
Figure 7. Overprovisioning with nested rules configured
Consult the HP 3PAR Priority Optimization white paper 18 for additional technical and usage information for the QoS product.
HP 3PAR Persistent Ports
The Persistent Ports (or virtual ports) feature minimizes I/O disruption during an HP 3PAR StoreServ online upgrade or
node-down event. HP 3PAR Persistent Ports functionality, provided in HP 3PAR OS 3.1.2, performs upgrades on a
node-by-node basis, transparent to host multipathing software, avoiding any path failovers. Persistent Ports also
protects applications against any possible outage resulting from a controller node failure.
Regardless of the cause of a controller node failure (upgrades, maintenance, or power failure), Persistent Ports enables a
seamless failover of all the ports on that affected node to their predefined ports on the Partner Node. From a host
standpoint, connections to HP 3PAR StoreServ continue uninterrupted, with all I/O routed through a different port on
HP 3PAR StoreServ. This helps to achieve an uninterrupted service level for customer applications running on
HP 3PAR StoreServ.
Persistent Ports technology leverages SAN fabric NPIV functionality for transparent migration of a servers connections
to a predefined partner port on the HP 3PAR StoreServ during software upgrades. A predefined, host-facing port on the
HP 3PAR StoreServ assumes the Identity of the host facing ports for the node being upgraded.
As a best practice for SQL Server environments, configure HP 3PAR Persistent Ports to maintain storage connectivity and I/O
continuity without having to endure multipath failover and failback disturbances. 19 This achieves smoother, consistent
solution functionality during maintenance and component failure situations.
18
HP 3PAR Priority Optimization white paper, document number: h20195.www2.hp.com/V2/GetPDF.aspx/4AA4-7604ENW.pdf
19
HP 3PAR VMware ESX Implementation Guide, bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c03290624/c03290624.pdf
17
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
For proper functionality of Persistent Ports, enable NPIV on the fabric switches if not already enabled, and verify the correct
zoning of array ports to the host. For additional information, refer to the HP 3PAR StoreServ Persistent Port white paper. 20
For Brocade fabrics, use the portcfgshow command to confirm the default NPIV functionality for the fabric switch ports used
by 3PAR storage.
For the HP 3PAR StoreServ, use the showport command, to find the partner port. Use the controlport
[failover|failback] N:S:P command to test array port failover and failback.
------------------------------------------------------------------------------------------------
20
HP 3PAR StoreServ Persistent Ports white paper, hp.com/h20195/v2/GetPDF.aspx%2F4AA4-4545ENW.pdf
21
HP 3PAR StoreServ Storage Concepts Guide,
h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?lang=en&cc=us&contentType=SupportManual&prodTypeId=12169
&prodSeriesId=5335712
18
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
To improve the performance of HP 3PAR 7200 for SQL Server environments, the growth increment should be the size of a
1024 MB stripe that spans all the physical disks in the CPG. The optimal growth increment depends on the RAID level, the
number of controller pairs (1 controller pair for the 7200), and the number of disks in the CPG. Calculate the optimal growth
increment as follows:
(the number of physical disks in the CPG) x (1024 MB) x (number of Data disks in the RAID set) / (number of disks in the
RAID set).
For SSD, it is best to set the Growth Increment based on the number of nodes in the system. On 2-node systems, set the
value to 8 GB, on a 4-node system to 16 GB, on a 6-node system to 24 GB, and on an 8-node system to 32 GB
Example with RAID 5:
Number of 10k rpm disks in the CPG = 64
Four shelves or cages contain the FC disks. This includes the controller shelf that also contains disks; therefore RAID 5
(3D + 1P) with cage redundancy is used (3 Data disks per a RAID set of 4 disks). If five shelves had been used to contain
the 10k disks, then RAID 5 (4D + 1P) could be used and still maintain cage level redundancy.
Optimal Growth Increment for this RAID 5 CPG =
(64 disks) x (1024 MB) x (3 Data disks in the RAID set) / (4 disks in the RAID set)
= 49152 MB or 48 GB
Example with RAID 1:
Use the same 64 disks as previously described (multiple CPGs can use the same disks), the same four shelves, and a
RAID 1 configuration with a RAID set of 2 disks.
Optimal Growth Increment for this RAID 1 CPG =
(64 disks) x (1024 MB) x (1 Data disk in the RAID set) / (2 disks in the RAID set)
= 32 GB
Set the growth increment using either the HP 3PAR CLI or the HP 3PAR Management Console (MC).
Adding disks to a CPG
While adding disks to a CPG does not add overhead in and of itself, subsequent CPG data load balancing across the newly
added disks may impact performance. To minimize the performance impact of load balancing operations for SQL Server
environments, it is best to add as many disks as possible, and in multiples that use all disk chassis used by the CPG.
For example, if a CPG spans four disk chassis, then add disks evenly to all four chassis at the same time. For the
HP 3PAR StoreServ 7000, this would mean adding a minimum of a pair of disks to each of the four chassis. However, if
economically feasible, it would be better to add twice that many disks at a time, which would reduce the load balancing
influence by spreading the load across a greater number of spindles.
HP 3PAR StoreServ and VMware vSphere hardware-accelerated block zeroing and zero detection
VMware has several file system configuration options. For the virtual servers that will host SQL Server or other applications
in the shared environment where VMFS will be implemented, use Eager-Zeroed Thick (EZT) virtual disks. Eager Zeroed disks
provide performance and security benefits over Lazy-Zeroed-Thick disks, whose blocks are zeroed on demand the first time
a block has data written to it. When using the EZT format during virtual disk creation, all the space is pre-allocated and
pre-zeroed from the vSphere perspective. This would normally be both time-consuming and resource-intensive; however,
HP 3PAR StoreServ avoids this issue. The vSphere VAAI block-zeroing primitive uses the SCSI WRITE_SAME command to
write zeros, which allows HP 3PAR StoreServ to keep volumes thin while speeding up the provisioning of VMs.
Note:
HP recommends that VMFS volumes should use EZT along with Zero Detect to maximize the performance and space
savings in TPVVs.
19
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Enable HP 3PAR zero detection with HP 3PAR Management Console by using the Advanced options checkbox in either the
Create Virtual Volume wizard or the Edit Virtual Volume wizard as shown in figure 8.
HP 3PAR zero detect is also effective in extending thin provisioning to the database by keeping allocated database storage
thin due to the zero fill nature of SQL Server data files.
Figure 8. Zero detect Advanced options check box
20
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
The expectation for a successful AO implementation is that SQL Server data access patterns will remain relatively the
same over time with a high degree of locality. If data access patterns are constantly moving over a large area, with no
real pattern, then AO is less effective.
Scheduled AO monitoring and migration cycles help maintain performance for slowly changing access patterns. AO has a
powerful scheduling interface. AO schedules can be as simple as a particular weekly period of time, or as complex as
specific minutes and hours, during specific days and months.
Plan for migrations to occur during non-peak usage times, because any I/O to the disks affects performance as expected
and migrations do incur disk I/Os.
Use the AO Performance mode setting for SQL Server.
Create VVs from the FC CPG tier, so new space allocations come from a good performance tier by default.
HP Storage Services can provide professional support for your HP 3PAR StoreServ environment, including proactive
advice, performance baselines, and deep performance analysis when you do not have the expertise in house, or need
an expert set of eyes to help ensure sound configuration decisions.
Figure 9 shows an HP 3PAR Management Console AO Region I/O Density report, based on 12 hours of data collected from a
single SQL Server instance with two active databases. Each database had its own HP 3PAR StoreServ data LUN. Logs were
located on separate LUNs, but not included in the AO configuration. One database had about 3 times the workload of the
other. They were both active at the same time, with an average response time of 15 ms.
The charts show the same regions from two different perspectivesSpace and I/O rate. The figure shows the Space to I/O
Density chart on the left and the I/O rate to I/O Density chart on the right. The combined charts show lots of space with little
I/O, and a little space with lots of I/O. This workload is a good candidate for AO if the I/O profile remains relatively stable over
time. If the lots of space with little I/O area remains relatively the same, then AO has already found those regions and can
move them to more economical storage, such as NL devices, freeing up the more expensive space for hotter data.
Figure 9. I/O Rate Density graphs of SQL Server workload that may benefit from 3PAR Adaptive Optimization
HP ProLiant Servers
The vSphere hosts are ProLiant BL460c Gen8 blade servers. The hosts contain the consolidated, virtualized
Windows Server 2008 R2 Failover Cluster Services environments.
HP ProLiant BL460c Gen7 blade servers host infrastructure applications and services. Examples of infrastructure items
include VMware vCenter Server, HP Systems Insight Control, Windows Active Directory, DNS services, and HP 3PAR
Management Client and Command Line Interface applications.
21
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Recommendation:
use the HP-provided VMware ESXi media when installing vSphere on ProLiant servers, this media includes all of the drivers
required for the installation.
The Microsoft Assessment and Planning (MAP) toolkit assists in evaluating existing SQL Server workloads to understand the
current SQL Server sizing requirements and ease migration planning. Use the MAP tool as follows:
With planning and growth estimates for new SQL Server 2012 workloads, to help form the total sizing requirements for
the array.
To help inventory different environments for planning, such as VMware, MySQL, Oracle, and servers running various
versions of the Windows Server operating system.
Normally, failback of cluster resources is de-selected; however, for load balancing across the vSphere hosts, failback was
set to Immediately. This is set based on site policies, but if failback is de-selected, then plan for manual load balancing
after a recovery operation.
22
MS WSFC with SQL Server, msdn.microsoft.com/en-us/library/hh270278.aspx
22
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
HP 3PAR StoreServ Storage is more than able to handle the I/O needs of your virtual consolidated environment when sized
correctly, which includes tuning the environment so that storage is used to service the hard I/Os that cannot be easily
handled by server cache.
Figure 10. Storage I/O during SQL Server memory tuning
23
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Item Comment
Eager Zeroed Thick (EZT) for 3PAR LUNs EagerZeroedThick format should be used for all vSphere VMSF disks using
3PAR storage (not applicable for RDMs)
Multipath Path Policy = Fixed Fixed is required for volumes used with MS Cluster Service.
Manually configure the Preferred path for evenly distributed path usage.
RDM Compatibility Mode Physical required for sharing between VMs on any server
SCSI Controller for RDMs not 0 Use SCSI 1 for example, not the default SCSI 0
vSphere DRS VM-Host Affinity rules used to keep VMs on separate hosts
23
Setup for Failover Clustering and Microsoft Cluster Service: pubs.vmware.com/vsphere-51/topic/com.vmware.ICbase/PDF/vsphere-esxi-
vcenter-server-51-setup-mscs.pdf
24
HP 3PAR VMware ESX Implementation Guide, Working with 3PAR StoreServ and VMware UNMAP to stay thin:
hp.com/bc/docs/support/SupportManual/c03290624/c03290624.pdf
24
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
For additional vSphere information, refer to KB 2014849, Using vmkfstools to reclaim VMFS deleted blocks on
thin-provisioned LUNs.
ESXi handling of SCSI Queue Full and Busy messages from 3PAR StoreServ
Configure ESXi SCSI Queue Full and Busy message handling on a per LUN basis per the implementation guide. 25 This setting
is important for Priority Optimization (QoS) functionality. Set both the QFullSampleSize and the QFullThreshold values.
HP recommends following the implementation guide for Queue Full and Busy message handling whether or not QoS
is implemented.
ESXi Perennially-Reserved RDM volumes with MSCS
Set perennially-reserved to true. Failure to do so may result in extremely long boot times for cluster reserved RDM devices.
Refer to KB 1016106 26 for implementation instructions.
25
HP 3PAR VMware ESX Implementation Guide, ESX/ESXi Handling SCSI Queue Full and Busy Messages from the
HP 3PAR StoreServ Storage Array
26
VMware KB 1016106: kb.vmware.com/kb/1016106
25
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
26
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Figure 12 shows an Adaptive Optimization I/O Density Report after AO migrated data and some time passed. This case
presents only two tierstier 1 and tier 2. Tier 2 is composed of NL devices and is intended to host the AO migrated data
that has no real I/O workload running on it. In this configuration, Tier 1 is sized to handle the I/O workload by itself, because
there is no Tier 0 to provide assistance. Response times before and after the migration remained around 15 ms, since there
was no Tier 0. The benefit of AO in this case is that over 2 TB of space was migrated off Tier 1 storage. The graph on the
right shows a small portion of the I/O workload being serviced by the NL CPG. This is expected as workload profiles shift
over time. Running subsequent AO schedules helps to keep active data on Tier 1 storage and maintain performance.
Figure 12. AO two tier results after a migration and some elapsed time.
Figure 13 shows an Adaptive Optimization I/O Density Report before a scheduled AO migration. In this case, AO has not been
run in a long time and workload dynamics have caused more I/Os to be serviced out of the NL CPG tier. Performing another
AO migration based on this data will migrate hot data regions back to Tier 1 storage and inactive regions back to Tier 2
storage. AOs built-in scheduling mechanism automates these tuning procedures and helps to optimize performance.
Figure 13. AO two tier results after a migration and some elapsed time.
27
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Figure 14 shows an Adaptive Optimization I/O Density Report with three tiers. The SQL Server environment consists of a
single instance with two databases. The Microsoft SQL Server Management Studio (MSSMS) Disk Usage report shows the
Data Files Space Usage to be 545 GB each, and over 85% allocated. Running AO in this environment dramatically changed
the entire database dynamics.
No longer was the CPU waiting 15 ms. for storage data I/Os to complete; the six CPU VM went from 50% CPU usage to 100%
(figure 15). Data LUN average response times dropped to under 2 ms. Perfmon showed that the effect was so dramatic that
Database Transactions/sec almost tripled. AO achieved these results by finding and migrating the active regions to SSDs,
while moving inactive regions to NL devices. This can be accomplished by manually moving the database files or the VVs to
SSD, but in that case, all the space, not just the hot regions, will be moved.
Figure 14. AO Region IO Density report after several hours of migration in a 3 tier environment.
Figure 15. CPU usage increased to 100% when AO moved heavily accessed data to SSDs
The preceding AO migration shows that placing growth limits on a CPG can be beneficial. Placing usage limits on a tier
prevents unlimited, uncontrolled tier usage, and helps with resource sharing. Setting growth limits on a CPG used in an AO
configuration provides quality of service by limiting data migration into a CPG tier. For this use case, about 853 GB of data
was migrated to Tier 1. Figure 16 shows the Region Density report when growth limits are set and only 455 GB of data is
migrated to SSDs. This configuration has average data LUN response times of below 5 ms, CPU usage running around 90%,
and over two times the database Transactions/sec above the earlier baseline measurements. The number of CPUs should
be tuned, but this test shows how AO can change the working dynamics of the SQL Server environment.
28
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Every environment is unique, with its own workload profiles and address space usage patterns. Users should analyze their
environments or consult with HP Services to discover how AO can support their performance goals.
29
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Use either the 3PAR CLI or the 3PAR Management Console to monitor and set QoS rules. Some helpful CLI commands are:
setqoscreate, modify delete QoS rules on VVsets (see figure 19).
showqosview existing QoS rules and configurations (see figure 20).
statqosprovides useful real time performance reporting (see figure 21).
Figure 20. showqos example output (shows QoS settings). Sorting on second column (starting at 0).
30
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
hp3par2230 cli%
Figure 22 shows a portion of the statqos output and the useful information displayed. In the figure, the major sections have
been divided and numbered from 1 to 7. This is a short explanation of those sections:
1. VVset name, which starts with the System wide VVset with the name all_others.
2. IOPS information. First is the statqos Quantity (Qt) setting followed by Current, Average, and Max values calculated over
the same time.
3. Throughput information. Notice that for vvset_qos85, the current IOPS is running at the set limit of 7000 IOPS.
Additionally, there was a momentary peak of 87,639 KB/sec, which was then quickly limited.
4. Service time is the total time to service the I/O, including both the QoS delay and the remainder of the system I/O
processing time.
5. Wait time in milliseconds is how long I/O was delayed strictly due to the QoS rule.
6. Size of the I/O request is the size of the I/O coming into the storage port.
7. Number of QoS rejected I/O requests (Rej), the average QoS queue length (Qlen), and the average wait queue length
(WQlen) are displayed. Qlen is the total queue length of both the I/Os delayed by QoS and the I/Os processed without
QoS delay. WQlen is the average number of I/Os delayed by QoS.
The vvset_qos85 VVset shows some interesting data. This is a VVset with a single VV; however, this VV is provisioned from a
CPG that is configured in a 3-Tier Adaptive Optimization configuration with SSDs. The AO graphs for this VV are shown earlier
in figures 14 and 15. SSDs are extremely fast, and in this case, the VV is servicing 7000 IOPS (limited by QoS).
Normally, do not use AO to boost performance, and then limit that performance with QoS; however, this example is
interesting for observing QoS behavior with AO. The columns labeled 4 and 5 show that Priority Optimization is currently
adding 3.3 ms of wait time (Wtt_ms) for vvset_qos85 I/O, for a current total service time (Svt_ms) of 6.7 ms. Column 2
reports the system is currently servicing I/Os at right about the QoS IOPS limit (7000 vs. 6995). This shows that back-end
devices can provide much more I/O, but are throttled by QoS functionality.
31
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Figure 23 shows a follow-on statqos report after tuning the vvset_qos85 QoS IOPS limit from 7000 to 9000. The IOPS have
settled to a current value of over 8500. The WQlen value is zero; however there are still some I/Os being limited, as shown
by the Wtt_ms value of 1.4 ms. Because this is a Tier 1 application with data on SSDs, the IOPS limit is still set too low. To
remove the wait time being generated for I/Os to the vvset_qos85 VVs, increase the IOPS limit further. This makes sense,
since this is a Tier 1 application using SSDs.
Figure 23. statqos example output with IOPS changed from 7000 to 9000.
Summary
Key features that distinguish HP 3PAR StoreServ and provide performance, ease of use, efficiency, and flexibility are:
HP 3PARs Thin Provisioning technology
HP 3PAR Priority Optimization (QoS)
HP 3PAR Dynamic Optimization
HP 3PAR Adaptive Optimization
HP 3PAR StoreServ is designed for use with VMware vSphere on ProLiant Servers, and WSFC; the four together provide a
combination for success.
The trusted VMware virtualized platform installed on HP ProLiant Servers provides a versatile base for consolidated
Windows Server Failover Clustering for MS SQL Server solution. VMware simplifies physical server provisioning and provides
a consistent virtualized hardware platform to build on.
MS Windows Server running WSFC on VMware vSphere behaved the same as solutions running directly on a ProLiant Server.
Consolidating multiple discrete SQL Server environments onto ProLiant Servers running VMware vSphere helps to simplify
management and increase overall storage performance.
The HP 3PAR StoreServ storage products performed seamlessly in the consolidated SQL Server environment, providing
performance and highly available storage in an easy to manage solution. HP 3PAR StoreServ is efficient, reducing data
duplication and eliminating the need for over-provisioning, while ease of management and implementation help reduce the
chance of configuration errors, and reduce the cost of ownership.
VMware vSphere on HP ProLiant Servers, running consolidated SQL Server solutions with HP 3PAR storage, provides a
winning solution for critical SQL Server business solutions.
32
Technical white paper | Best Practices for Implementing SQL Server with HP 3PAR StoreServ and VMware vSphere
Resources
HP Storage Arrays
HP Storage
HP Storage Solutions for MS SQL Server
HP Insight Control Storage Module for VMware vCenter Server
HP Servers
hp.com/go/Servers
Learn more at
hp.com/solutions/activeanswers
Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for
HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as
constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.