HP StoreAll Storage 6.2.

3 Release Notes

HP Part Number: AW549-96065

Published: March 2013
Edition: 8
 Copyright 2013 Hewlett-Packard Development Company, L.P.
Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation.
 Version:
6.2.3 (build 6.2.407)

Description
Update eligibility
Customers running HP StoreAll OS Software 6.2.x, 6.1.x, 6.0.x, 5.6.x, 5.5.x, or 5.4.x are eligible
for the update. Refer to the administrator guide for your storage system for specific update requirements
and procedures. Customers running IBRIX OS Software 5.3.2 or earlier versions should contact HP
Support to determine compatibility before updating their software.
Supersedes
6.2.2 (build 6.2.386)
nl

Product models
HP StoreAll OS Software

Devices supported
HP StoreAll 9300 Storage Gateway
HP StoreAll 9320 Storage
HP IBRIX X9720 Storage
HP StoreAll 9730 Storage

Operating systems
StoreAll 9000 supported devices use the Red Hat Enterprise Linux 5.5 (64 bit) operating system.

StoreAll Linux and Windows Client Support

The HP StoreAll OS 6.2.3 release supports StoreAll Linux and Windows clients version 6.2.1. If you
are already running the StoreAll 6.2.1 clients, you do not need to upgrade the client. However, StoreAll
OS 6.2.3 includes performance fixes for both Linux and Windows clients. If you are experiencing
performance issues, upgrade the Linux or Windows client to version 6.2.3.

Description 3
Other supported software

Software Supported versions

Linux 9000
nl
Red Hat Enterprise Linux 5.2, 5.3, 5.4, 5.5, 5.6, 5.7, 5.8, 6.1, 6.2 (all 64 bit)
clients
SUSE Linux Enterprise Server 11 (64 bit)
SUSE Linux Enterprise Server 10 SP3 (64 bit)
openSUSE 11.1 (64 bit)
CentOS 5.2, 5.3, 5.4, 5.5, 5.6, 5.7, 5.8, 6.1, 6.2 (all 64 bit)

Windows 9000 Windows 2008 R2 (32 and 64)

clients
Windows 2008 (32 bit and 64 bit)
Microsoft Windows 7 (32 bit and 64 bit)
Windows 2003 SP2 (32 bit and 64 bit)
Windows 2003 R2 (32 bit and 64 bit)

SMB clients Windows 2008 R2 (64 bit)

Windows 2008 (32 bit and 64 bit)
Windows 2003 R2 (32 bit and 64 bit)
Windows 2003 SP2 (32 bit and 64 bit)
Windows Vista (32 bit and 64 bit)
Microsoft Windows 7 (32 bit and 64 bit)
Windows XP (32 bit and 64 bit)
MAC 10.5, 10.6 and 10.7

Internet Protocol IPv4

iLO firmware iLO2 2.12 for G6 servers

iLO3 1.28 for G7 servers

Browsers for Microsoft Internet Explorer 8 and 9 (Windows and Linux)

management
nl

Firefox 16, 17, 18, and 19 (Windows and Linux)

console GUI
Chrome v18 (Windows)
Adobe Flash Player 9.0.45 or higher for viewing the charts on the GUI dashboard

C7000 OA firmware 3.60 (July 2, 2012)

NOTE: StoreAll has dropped support for NDMP with Netbackup.

NOTE: For the latest compatibility information, see the HP StoreAll Storage Compatibility Guide on
HP Storage SPOCK:
1. Go to http://spock.corp.hp.com/spock/index.aspx.
2. In the left navigation pane, select Storage Servers (NAS) under Other Hardware.
3. Select HP StoreAll Storage Compatibility Guide in the right pane.

Updated firmware included in the Firmware Management Tool (FMT)

1210m controller firmware version 1.76 (9730)
6GB SAS Switch firmware version 3.5.1.0 (9730)
ExDS9100cc firmware version 1.40 (X9720)

4 Updated firmware included in the Firmware Management Tool (FMT)

Languages
International English

Enhancements in the 6.2.3 release

The intent of the 6.2.3 release was to resolve key issues found in earlier releases. See Fixes (page 7)
for a list of those fixes.

Enhancements in the 6.2.1 release

The 6.2.1 release provides the following enhancements.
New Feature/Capabilities/Solutions
StoreAll REST API The StoreAll REST API provides the ability to upload and download files,
assign custom (user-defined) metadata to files and directories, manage file retention settings,
and query the system and custom metadata of files and directories. You can associate any
number of custom (user-defined) metadata attributes to any file or directory stored on a
StoreAll retention-enabled file system where the Express Query metadata store is enabled.
Each custom attribute consists of an attribute name and assigned value. The API provides
commands to create custom metadata entries for a file, replace values of existing entries,
and delete entries. The API extends the existing HTTP Shares feature that was introduced in
IBRIX OS version 6.0.
Express Query Express Query provides a per file system database of system and custom
metadata, and audit histories of system and file activity. When Express Query is enabled on
the file system, you can manage the metadata service, configure auditing, create reports
from the audit history, assign custom metadata and certain system metadata to files and
directories, and query for selected metadata from files.
Installation Enhancements
Installation wizard for 9730

Firmware version verification

Storage provisioning verification for 9320

Support Enhancements
Server hardware monitoring support for the 9300 Storage Gateway platform.

Firmware Management Tool for the 9320 and 9720 Storage platforms.

Server and Storage hardware monitoring support for the 9320 Storage platform.

Failed node crash capture management Failed Node iLO Crash Management enables
customer to collect the core dump of the failed node (Segment Server) when Fusion Manager
initiates the failover. This functionality is not enabled by default.
Troubleshooting Guide enhancements
Manageability
Scheduling for antivirus scans

New or revised wizards:

Expansion wizard
High availability wizard
Quota management wizard

Languages 5
 Rebalancing wizard
Segment evacuation wizard
Data tiering wizard
Replication wizard and health checks

Implementation changes in the 6.2.1 release

The following implementation changes occurred with hardware monitoring:
During an upgrade, user must now run the Firmware Management Tool and perform a set of
manual steps to update the firmware.
All of the logical (volumes) and physical (controllers, ports, drive enclosures, drives, etc.) objects
that are under the control of a storage controller or pair of storage controllers are grouped together
within a new logical STORAGE_CLUSTER component in the storage health report to describe their
relationship in the storage topology.
On the 9320 Storage platform, the HA_STORAGE health monitoring category has been replaced
with the new STORAGE_CLUSTER category for monitoring couplet visible storage. The
HA_STORAGE category is still supported but is being deprecated in the next release.
A subset of all visible storage clusters can be monitored by specifying a list of storage cluster
UUIDs on the hpspmon command line. Prior to this change all storage clusters would have been
monitored.

File systems are created in 64mode by default

Prior to version 6.1, file systems were created by default with 32-bit compatibility mode enabled. Starting
in 6.1 the default behavior has changed and compatibility mode is now disabled. The StoreAll OS
software creates file systems in 64-bit mode unless you changed the compatibility mode when creating
the file system. However, if the original file system was created with 32-bit compatibility mode enabled
and you later upgrade to 6.1 or later and then extend the StoreAll file system, new segments will be
formatted into the file system as 64-bit mode segments.
The following output shows the relevant portion of the output from the ibrix_fs -f fsname -i,
indicating how newer segments (37,38,39) appear on a file system originally created with compatibility
mode enabled.

NOTE: The 9th column (FFREE) which is your total available inode count per segment for the
original segments 66 Million per segment and that of the newer 64 bit segments of 1 billion per
segment. This segment mix and inode count does not negatively affect the operation of your file system
nor any applications.

Software snapshots
Upgrading pre-6.0 file systems for snapshots
To accommodate software snapshots, the inode format was changed in the 6.0 release. Consequently,
files used for snapshots must either be created on StoreAll OS software 6.0 or later, or the pre-6.0 file
system containing the files must be upgraded for snapshots. To upgrade a file system, use the
upgrade60 utility. For more information about the utility, see upgrade60 in the HP StoreAll Storage
CLI Reference Guide.

6 Enhancements in the 6.2.3 release

 IMPORTANT: This tool requires exclusive access to the file system and implies down time, as the file
system must be unmounted before the upgrade.

Restrictions for rename options

For file systems that were created in a release earlier than 6.0 but have not been upgraded, StoreAll
OS software can preserve all name space data in snapshots but cannot preserve file data for objects
(files). To help prevent hybrid snap trees, in which a snap tree contains objects with the old format,
restrictions have been implemented on rename operations. The following restrictions apply to hybrid
file systems:
Only directories created in version 6.0 or later can become snap tree roots.
If the old directory is not in a snap tree and the new directory is in a snap tree, rename is allowed
only if the object being renamed is snapable (that is, it has the new inode format).
The following restrictions apply to both hybrid file systems and pure 6.x file systems:
A snap tree root cannot be renamed. Also, the path to a snap tree root cannot be changed.
Rename is allowed when neither the old directory or the new directory are in snap trees.
Rename is allowed when the old directory and the new directory are in the same snap tree.
Rename is not allowed when the directories are in different snap trees.
These restrictions are intended to prevent hybrid snap trees containing files with the old format. However,
hybrid snap trees can still occur when a directory having the new format is populated, using rename,
with old format objects and that directory is then made into a snap tree root or is renamed into a snap
tree. The StoreAll OS software does not prevent this situation because it could take a prohibitively
long amount of time to perform a complete scan for old objects in the sub tree being moved if the new
sub tree was sufficiently large.

Fixes
Fixes in the 6.2.3 release
The firmware bundle includes new firmware version 1.40 for the X9720 storage controllers
(ExDS9100cc). Version 1.40 contains critical fixes and is a mandatory upgrade. After updating
the storage controllers to version 1.40, it may take 10-12 minutes for the storage controllers to
complete a reset and restart with the new firmware. During this time, issuing the command
hpsp_fmt --flashrec for the storage_ctlr component may report missing devices. Users
should wait until both controlls restart (with the new firmware) before performing any other
operations on these controllers.
Fixed an error counter issue (the number of errors did not match the number of corrections) in
ibdircheck. Now, the error counter is increased once (instead of twice) when ibdircheck
finds and fixes a problematic dentry.
ibdircheck no longer deletes a freshly-resolved migrated hard link. Previously, if a problematic
dentry was successfully resolved by lookup, ibdircheck did not check the dentry again and
the dentry may be accidentally deleted.
During inode creation, the inode is now checked to see if it is a replica. This prevents inode
creation from failing due to the inode quota limit being exceeded.

Fixes 7
 Some SNMP managers, such as Netcool Omnibus Manager, received StoreAll SNMP traps with
an incorrect Specific Trap type and mismatched variable bindings.
The fix changes the SNMP Trap OID (OID of IBRIXGENERICEVENT) to .1.3.6.1.4.1.18997.2.0.1
and adds eventIndex to the variable bindings. Additionally, this fix includes a change to the
StoreAll MIB file, which requires the following configuration steps after installing the 6.2.3 patch:
1. On the CMS server, you must compile and manually register the StoreAll MIB file using HP
Systems Insight Manager 7.1:
a. Download ibrixMib.txt from /usr/local/ibrix/doc/ on the StoreAll server.
b. Rename the file to ibrixMib.mib.
c. In HP Systems Insight Manager, complete the following steps:
1) Unregister the existing MIB using the following command:
<BASE>\mibs>mxmib -d ibrixMib.mib
2) Copy the ibrixMib.mib file to the <BASE>\mibs directory and execute the
following commands:
<BASE>\mibs>mcompile ibrixMib.mib
<BASE>\mxmib -a ibrixMib.cfg

2. Once the new MIB file is successfully compiled and loaded, you must apply the
ibrixSnmpEvent.sql in HP Systems Insight Manager 7.1:
a. Download ibrixSnmpEvent.sql from usr/local/ibrix/doc/ on the StoreAll
server.
b. In HP Systems Insight Manager, perform the following steps:
1) Copy the ibrixSnmpEvent.sql file to <BASE>\config\preload\71\
addfiles\.
2) Execute the following command from CLI:
<BASE>\mxconfigrefresh

NOTE: For more information about loading the MIB, see the Compiling and customizing MIBs
chapter in the current Systems Insight Manager User Guide, which is available at: http://
www.hp.com/go/insightmanagement/sim/.
If HP Systems Insight Manager is shared between StoreAll clusters, you must install the 6.2.3 patch
and complete these steps on all StoreAll clusters.

Fixed an issue that caused files to show as duplicates on an NFS share.

Unexpected server restarts have been fixed by changing how ibr_dcn_notify_all addresses
delegations.
Fixed the code that checks for quota initialization to prevent a kernel panic that could occur when
disabling and enabling Directory Tree quotas.
After unmounting and remounting a file system, the create time would be listed as 0 (zero). Fixed
the code so the create time check occurs after the format version check.
A segment server routine within ibdircheck was changed to only perform the check for which
the routine was originally intended. This reduces performance issues and the possibility of causing
issues on healthy file systems.
The online upgrade script was fixed to prevent the upgrade from failing when the file system was
unmounted, which previously resulted in no error being displayed and the system being partially
upgraded.
Fixed a memory leak and performance-related issues when running StoreAll Windows Client for
long hours.

8 Fixes
 Running replication with antivirus scanning no longer corrupts ibrmode, which previously made
the scan appear not to occur.
When performing a segment evacuation on a shadow directory (due to a bad segment), the
evacuation no longer fails.
Previously when hard links were created on a StoreAll file system from the StoreAll Windows
Client, unmounting the file system might result in a failure. The fix ensures the failure no longer
occurs.
The file name length value was corrected in the Notify response packet to prevent the SMB client
from failing during create, change, or delete operations on files or folders.
The HP-SMB Srvsvc service no longer fails due to an excessive number of RPC connection
requests.
Access-based enumeration now works for all types of directory enumeration requests for SMB
clients.
After an upgrade on the StoreAll 9730, a simultaneous reboot of multiple servers could cause
the SAS expanders in MDS600 and D6000 disk enclosures to remain undiscovered. This resulted
in the hard disk drives not appearing and the disk enclosure failing to come online, even in a
degraded mode. This has been corrected with an upgrade to Virtual SAS Manager firmware
version 3.2.2.0.
After archiving files, segment evacuation no longer fails and an RO lock error no longer occurs
when accessing files on another node.
Significantly improved sequential read performance for HP-SMB (up to four times faster compared
to previous releases).
The HP-SMB service called lwiod would restart due to an assertion failure, which occurred when
a client attempted to recover a previously opened session after a disconnect/connect failure.
The HP-SMB share management utility called lwnet previously failed when run with the lwnet
share --set-info option. This occurred only when a domain user was used in the command,
and the node had been removed from the domain.
The HP-SMB service lwio failed when multiple clients accessed files or folders under a share and
simultaneously an application like PowerShell subscribed to event notifications on a shared folder.
When using the ibrix_avquarantine command, if the complete path length for the infected
file exceeded 200 characters, a crash occurred and the infected file was not moved.
Fixed the order of components during an upgrade so that creating archiving-enabled file systems
post-upgrade no longer requires a restart of the Fusion Manager component.
LUN discovery was corrected to handle multiple comma-delimited vendor storage names.
Fixed an issue that caused i_blocks and i_total_blocks to become out of sync when a directory
was being created.
Refresh stale dentry now displays when a misidentified dangler or a legitimate corrupted
directory condition occurs.
A missing attribute in the LDAP server user record now generates an error in the log, but no longer
results in a crash.
Wrapper scripts were modified to accept spaces in arguments.
The IBFS driver registry entry is now stored in memory to avoid the overhead of reading from the
registry.
Fusion Manager correctly redirects stdout and stderr messages to the log file, which ensures
that phases 2 and 3 of the ibrix_fsck command will complete.
The file name in the DCN response was changed to be the relative Windows file name (which is
the Windows server behavior) instead of the absolute UNIX file name.
Fixes 9
 The default number of adm server threads was increased from 10 to 24. Previously, adm server
threads were used only for Fusion Manager and IAS management but many utilities now use adm
server threads.
A race condition could occur in the StoreAll kernel when a delegation thread was deallocated
without being properly synchronized with other threads. This occurred only in very rare
circumstances. The fix ensures that proper synchronization occurs before performing the thread
deallocation.
An electrical signal on the cache module is now synchronized before being analyzed. Previously,
electrical noise on this signal had been driving the supercapacitor charging logic into an undefined
state.
A deadlock in the switchs internal thread handling was eliminated.
The controller firmware now retries certain operations that were prematurely marked as failed.
A reset signal on the cache module was previously being cleared prematurely in brownout
scenarios; this has been corrected.
The firmware was corrected to ensure that drive heads are safely parked during a loss of power.
Previously, the switch firmware was miscounting SAS frame errors, resulting in a generic warning
being displayed on the SAS switch.
The Firmware Management Tool now displays a more informative error message when it is unable
to retrieve a list of file serving nodes from the Fusion Manager.
The firmware bundle includes disk drive firmware version HPD3 for disk drives MB1000FBZPL
and MB2000FBZPN. HPD3 prevents an infrequent condition that may occur when the WRITE
SAME command is used during RAID ARRAY parity initialization, which can result in unexpected
data on the hard drive.
Fixed an issue with StoreAll Linux clients running Red Hat Enterprise Linux 6.x in which the system
froze for five seconds during file editing.

Fixes in the 6.2.2 release

Administrators were unable to add a user as nasxxx in SMB Authentication.
Share administrators were unable to add users and groups with parenthesis and ampersands in
their name to an SMB share.
Thousands of /tmp/tmp*.lvmtool directories were being created.
StoreAll nodes were not discoverable when the Fusion Manager IP address was used in HP Systems
Insight Manager (SIM).
A crash would sometimes occur while renaming a folder.
Quotas were being incorrectly reported after an upgrade.
The ibrix_fsck command was showing "illegal inode number while reading inode."
The ibrix_vs -a command would fail on combined 9730+9720 and 9320+9730 clusters.
Long delays (approximately 10 minutes) followed by an HTTP status code in the 5XX range would
occur while using the StoreAll REST API.

Fixes in the 6.2.1 release

There is no longer a 100 LUN limit when creating a file system.
When a blade was pulled out, the segment server might not failover.
The summary page does not show relevant message of the action taken by the user. The UI has
since been modified.

10 Fixes
 Request for a method for setting the pasv_address parameter in the vsftpd.conf file. This
parameter can now be set as follows:
CLI. Enter the following command, which will write the pasv_address parameter
(pasv_promiscuous=YES) into the sftpd configuration file:
ibrix_ftpshare -m ftp_share1 -c profile1 -I 10.9.60.61:21 -S
"allow_nat=true"

GUI. A new field named Allow NAT connections has been added to the FTP share modification
window.
Fusion Manager (FM) restarted a CFR job even if it is marked stopped by the user. Fusion
Manager has been modified so that it does not automatically restart a CFR job when the user
requests the CFR job to be stopped.
A soft lockup of the CPU was occurring during connection/delegation recovery.
The creation of directory tree quota names with a colon (:) in the name produced undesirable
results and it further impacted the creation of new "normal" directory tree quota names until the
entry with the colon character was removed. Users are now told in the HP StoreAll Storage File
System User Guide that they cannot create a directory tree quota containing a comma (,) or colon
(:) character.
When performing a "graceful" shutdown (ibrix_umount) of the StoreAll system for maintenance
reasons, the segment server hosting the root segment took 30 minutes per NFS export to timeout
and finally unmount.
When accessing StoreAll file systems through NFS, the client NFS access would hang intermittently.
When segments reached 7080% of capacity, clients were unable to write to an SMB share.
The quota current size would be reduced after running the rebalance task, even though files were
not modified or deleted. During movement of files across segments, there are intermediate replica
files that are created. Once data from an original file is synced with its replica, the original file
is deleted and the replica is promoted as the original file. These replicas have to be marked with
the directory quota ID, which is the same as that of the original master file. That was not occurring.
As the original file was deleted post replication, the directory quota accounted for it was getting
decremented. Since the newly created file was not marked with the directory quota ID, it was
never was accounted for. As a result, a reduction in quota usage was shown post replication.
The fix was to mark the replica created with the quota ID same as that of the original file, so any
data written into this file is accounted for against the set quota ID.
Data tiering could result in zero length files.
When a snapshot was taken, files could be deleted from the source directory.
Stack overflows were occurring, due to some of the CLI commands, such as ibrix_fs, requiring
a large buffer. The fix lessened the risk of a stack overflow.
Dangling dentries were created when mv and cp commands were killed during the file creation
operation.
The upgrade60.sh upgrade script was sometimes showing an invalid file system message in
environments where clusters were on a flat network.
A delay would occur between serial sequential file writes, about 2 to 20+ seconds between the
first file transfer and the next. This delay would be seen during the drag and drop of multiple 4
GB or larger files from Windows Explorer onto the StoreAll SMB share.
Checksum errors were being reported while accessing multiple database files from SUSE Linux
NFS Clients, which were reading multiple database pages (8 KB) into the NFS Client memory for
a SAP application. Although some of the data inside the 8 KB page was missing, when the same
page was re-read, the page data was completely intact and the checksum was correct. This

Fixes 11
 situation only occurred when the read operations involved a cluster network hop to a remote FSN
on a remote couplet, and the issue has been resolved.
During a mkdir operation, the xdr_encode_netobj dereferencing NULL
xdr_netobj->data pointer message was displayed. The underlying issue has been resolved
so the message no longer appears.
During Directory Change Notification (DCN) processing, a serialization issue could cause the
thread to loop forever while traversing the list of consumers of the notification. The Linux kernel
would subsequently detect a soft lockup and log messages in regards to the situation would appear
in the kernel log (BUG: soft lockup CPU#<X> stuck for 10s!
[i_worker_X:<pid>]). The serialization issue has now been fixed.
There was a request to document file system mount information (specifically, the change to the
default mount options in 6.1 and the use of ibrix_mountpoint to set mount options). The man
pages for the ibrix_mount and ibrix_mountpoint commands and the HP StoreAll Storage
File System User Guide were updated.
The StoreAll Windows Client would crash when the client was built in debug mode.
The following fixes have been made for the 9730 Storage platform:
Storage could become unresponsive when multiple surface scans were launched on a single
physical disk. Interaction between background surface scans and performance has been
significantly improved.
The controller caching policy has been adjusted to significantly reduce the occurrence of
command timeouts and aborts when running heavy workloads with small block sizes.
Drive LEDs could go dark when an MDS600 storage enclosure I/O module was
removed. Drive LEDs are now refreshed in this scenario.
Sometimes MDS600 storage enclosure I/O modules were not visible to the storage controllers
after a FW flash of the I/O module. This has been corrected.
Improved resilience against SAS topology disruptions due to events such as booting of blades,
which could lead to missing paths to storage.
For a list of firmware fixes, see the FMT User Guide (contact HP Support for more information).
MDS600 and D6000 drive enclosure drawers are now reported by number: Drawer 1 for the
front-facing left drawer and Drawer 2 for the front-facing right drawer.
A memory leak was occurring in Fusion Manager.
For a list of fixes in the Storage Provisioning Tool (SPT), see the HP SPT User Guide (contact HP
Support for more information).
The var/log/message file was not getting rotated nightly.

Workarounds
This section contains workarounds for product situations that may occur:

Installation and network configuration

On X9720 systems with G6 blades, the Quick Restore installation reports that the G6 Ethernet
firmware must be upgraded. The Ethernet device (eth0) must be up when you upgrade the
firmware. Use the following command to bring up eth0 and then upgrade the firmware:
# ifup eth0
The following document provides additional detail for HP Support personnel:

12 Workarounds
 http://saw.cce.hp.com/km/saw/view.do?docId=emr_na-c03466768
When creating a user NIC with a VLAN tag on a bond, running the ibrix_nic -c command
causes the parent bond to become degraded. The workaround is to determine the device that is
down and then bring up that device manually. For example, if bond0 is made up of eth4 and
eth5, use ifcfg to determine which device is down. Then use ifup to bring up the device,
such as ifup eth4.

Upgrades
The supported upgrade path from 5.6 or earlier is to first upgrade to 6.1. Before upgrading from
5.6 or earlier to 6.1, ensure that the NFS exports option subtree_check is the default export
option for every NFS export. The no_subtree_check option is not compatible with the StoreAll
OS software.
To add the subtree_check option, perform the following steps:
1. Unexport NFS exports:
ibrix_exportfs h <HOSTLIST> -p <CLIENT1:PATHNAME1>
2. Create NFS exports with the subtree_check option:
ibrix_exportfs f <FSNAME> -p <CLIENT1:PATHNAME1> -o subtree_check

NOTE: Multiple options can be specified using the o parameter by separating each option
by a comma (for example, -o rw,subtree_check).

3. Complete steps 1 and 2 for every NFS export.

4. Verify that all NFS exports have the subtree_check option set:
ibrix_exportfs l

If you plan to do an online upgrade, I/O must be occurring on a user virtual interface (VIF). Online
upgrades are only supported when I/O to shares is exclusively through User VIFs. When a VIF
is used, the server failover process sends the VIF to the running/active node. If I/O has occurred
through a non-virtual IP for the server, this I/O might need to be halted, and you might need to
reboot the server to proceed with an online upgrade.
After upgrading to 6.2.3, use the Firmware Management Tool (FMT) by entering the hpsp_fmt
fr server) command to determine if any firmware upgrades are required. If you are shown
the following message, proceed with the workaround:
ERROR: Could not recommend any flash action for one or more devices of
the following component type(s):
--------------------------------------------------------------------------------
Device :NIC
Code :0x1A0E
ERROR :Discovery failed for HP Embedded Network Adapter using
CP017785.scexe. Check log file
/var/log/hp/platform/fmt/firmware.log for more
information
--------------------------------------------------------------------------------
To resolve the issue:

Workarounds 13
 1. Manually check your version by entering the following command:
ethtool -i eth0
The command displays the following:
driver: be2net
version: 4.1.402.6
firmware-version: 4.1.402.20
bus-info: 0000:02:00.0
Bootable ISO Download:
2. Verify the firmware-version is 4.1.402.20. If it is not, download a bootable ISO file
(OneConnect-Flash-4.1.402.20.iso) from the following link to perform the NIC
upgrade offline:
http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareDescription.jsp?lang=en&
cc=us&prodTypeId=3709945&prodSeriesId=4085948&prodNameId=4085929&
swEnvOID=54&swLang=8&mode=2&taskId=135&swItem=co-106538-1

The HTTP WebDAV share might not let you create a file or folder with the following message:
You dont have permission to do this action.
To resolve the issue, remount the WebDAV share by entering the following command:
net use * http://192.168.1.1/smita/
In this instance, the HTTP WebDAV share is 192.168.1.1/smita.
The automated offline upgrade appears confused by clusters with more than 10 nodes. This issue
results in the upgrade failing when it attempts to connect via SSH to a filename rather than to a
server.
To resolve the issue, move the Active Fusion Manager (FM) back to the initial node by using the
following process:
1. From the current Active FM, disable failover across the cluster by entering the following
command:
ibrix_fm -m nofmfailover -A

NOTE: The above command does not place the local/Active FM in nofmfailover mode.

2. Make the desired node passive by entering the following command:

ibrix_fm -m passive -h <nodename>
3. Wait 1 minute for the commands to complete.
4. Place the current Active FM into 'nofmfailover' mode, this will trigger the migration by entering
the following command:
ibrix_fm -m nofmfailover
5. Wait another minute, after which, you should find the Active FM on the desired node. Verify
FM is at the correct node by entering the following command:
ibrix_fm -i
6. From the Active FM, place the rest of the nodes back into Passive mode by entering the
following command:
ibrix_fm -m passive -A
During an upgrade from 6.1 to 6.2.3, if there is I/O work load on a file serving node, the HP-SMB
services might not start itself after upgrade. This could make the SMB share not accessible from
that node. To recover from this situation, restart the HP-SMB services after upgrade if there is a
share access issue after upgrade.
14 Workarounds
 If clients previously mapped an SMB share using the hostname/FQDN, they will be prompted
continuously to enter their credentials when attempting to access the SMB share after the upgrade.
(The share can be accessed successfully using the IP address.) To workaround this situation, disjoin
and then rejoin all file serving nodes to the Active Directory domain. This can be done using the
GUI or the ibrix_auth command.
The 10Gb NIC driver is updated during the StoreAll OS 6.2.x software upgrade. However, the
new driver is not utilized/loaded until after the server has been rebooted. If you run the upgrade
firmware tool (hpsp_fmt) before you reboot the server, the tool detects that the old driver is still
being used.
Use of StoreAll Native clients for any SUSE or SLES kernel is not supported for file systems with
Express Query enabled.
After upgrading to 6.2.3, the segment evacuation task might fail with the message Completed
with error. This could occur in a rare situation when the file system only has two segments
and both the source and destination segments contain the file and its replica. To resolve this issue,
add another segment and restart the evacuation process.

Express Query Database

If you use hard links, be aware that there could be some inconsistencies in the metadata stored
in the Express Query database. If you perform any file system operations on a hard link that
changes the system metadata, such as the last modified time, all hard links on the file system will
show the updated system metadata on the file system as expected. In this example, the "stat"
command will show the correct last modified time for all hard links to the same file, since it shows
metadata as stored on the file system. However, only the hard link you modified will have its
system metadata updated in the Express Query database. The metadata stored in Express Query
for all other hard links to the same file will not show an updated last modified time in this example.
Express Query will treat all hard links as if they are separate files.
To find all hard linked files in a directory tree, enter the following command:
find <path> -links +1 -xtype f
Soft links, also known as symbolic links, do not have the these inconsistencies with the metadata.
Soft links are treated as "normal" paths by Metabox. Metadata for the soft link is maintained
independently of the target of the link. System metadata for the soft link is independent of the
system metadata of the target. Custom metadata is also independent. You can assign custom
metadata to a soft link that differs from the custom metadata assigned to the target of the link.
The following Express Query audit event types are not yet audited when data validation scans
start and end, even when enabled.
val_scan_started

val_scan_ended
The audit event types are in place to support this functionality in the future.
In the StoreAll REST API, the system::tier attribute is populated correctly when the file is first
created, or when it is first added to the metadata database if Express Query is enabled on an
existing file system. However, when the file system is moved to another tier by StoreAll file migration
utilities such as data tiering or rebalancer, that change does not yet update the system::tier
attribute available for query through the REST API.
Root segment evacuation fails on a file system that has express query and retention enabled.
Before running segment evacuation, disable Express Query by entering the ibrix_archiving

Workarounds 15
 S command. You can re-enable Express Query after segment evacuation completes. To enable
Express Query, enter the ibrix_archiving s command.
In certain cases where files created by MDExport or ibrix_audit_reports are very large, in the
millions of files range, the MDImport tool cannot successfully import the files. To allow the files to
be imported, break up the output files into smaller files, on row (line) boundaries, and use the
MDImport tool to import each of the files.

Management console
If you need to make cluster configuration changes such as adding, extending, or deleting a file
system, be sure that at least two file serving nodes in the cluster are UP and are not in nofmfailover
mode. If a configuration change is necessary and only one node is UP, save a copy of /usr/
local/ibrix/tmp/fmbackup.zip after each configuration change is completed. In the case
where only one node is UP, this copy can be used later if that node goes down before any of the
other nodes come up. In general, before you bring up cluster nodes that have been down for
some time, ensure that the cluster already has an active Fusion Manager running.
When the active management console is moved to nofmfailover mode, a passive management
console will transition to active mode. Be sure that this transition is complete before you move the
previously active management console from nofmfailover mode to passive mode. (Use the
ibrix_fm -i command to check the mode of each management console.) If the passive
management console has not yet assumed active mode, the management console being moved
from nofmfailover to passive mode will become active again.
File system creation on the 9300 server fails from the Getting Started Wizard. No data loss or
corruption occurs, and you can still create the file system using the Create Filesystem Wizard or
its comparable CLI command (ibrix_fs -c -f...).
File system creation from the Management Console, not the CLI, might sometimes fail with the
message Exporting filesystem (NFS) failed. The file system was created correctly and
all the other steps in the process have completed.
After creating a file system, you can create an NFS share and export the file system using either
the GUI or CLI, as described in the HP StoreAll Storage File System User Guide.

Fusion Manager
Actively running tasks (Fusion Manager event-based) stop reporting status to the active Fusion
Manager (FM) if an FM restart occurred on the system doing the monitoring for that task.
To resolve the issue:
1. Manually delete the task by entering the following command:
ibrix_tasks -k -n <taskid> -F
2. If the FM process was stopped or restarted while a task was running, the common user
database (CUD) and the specific task coordinator (or other task specific process) were left
running. When FM is restarted, it is no longer aware of these processes. A new task of the
same type might not start because the rogue task is still running. Remove the rogue task by
manually stopping the coordinator/task specific processes and restarting the CUD.
Under some circumstances the flashing of some hardware components can fail. See the HP FMT
User Guide for details (contact HP Support for more information about this guide).
If Fusion Manager (FM) is taking a long time to start and the logging for FM is showing APPARENT
DEADLOCK messages, purge the event database.
To purge the event database:

16 Workarounds
 1. Enter the following CLI command:
ibrix_event -p [-o DAYS_COUNT]
If you want to change the number of days the event database is purged, include the -o
DAYS_COUNT option when you enter the command.
Events are removed starting with the oldest events. The default is to remove the oldest seven
days of messages. Purge the event database periodically to prevent it from becoming too
large.
2. After the events are purged, create a new FM Store archive by entering the following
command:
ibrix_fm -B
3. Wait for a few seconds to let the Store archive propagate through the cluster and then restart
the active FM.
The credentials management service (hpsp_credmgmt) will not allow adding a credential for a
new MSA storage array after the credential database is created. The workaround is to install the
credentials manually.
To install the credentials manually:
1. Enter the hpsp_couplet_info -i command to get a list of UUID for attached MSA
enclosures. Each enclosure is on a separate line, as shown in the following example of the
command and its output:
[root@ib25-02s1 security]# hpsp_couplet_info -i
500c0ff1390c0000
2. Install the credentials for a single MSA by running the following four commands once for
each MSA that the hpsp_couplet_info command lists in step 1. Replace
<MSA_SAS_BASE_ADDRESS> with one of the UUIDs as provided by the output of the
hpsp_couplet_info command.
hpsp_credmgmt --update-cred -s
couplet:couplet/array/<MSA_SAS_BASE_ADDRESS>/mgmtport_a/rw
--cred-type upwpair --cred-username manage --cred-password '!manage'
hpsp_credmgmt --update-cred -s
couplet:couplet/array/<MSA_SAS_BASE_ADDRESS>/mgmtport_a/ro
--cred-type upwpair --cred-username monitor --cred-password
'!monitor'
hpsp_credmgmt --update-cred -s
couplet:couplet/array/<MSA_SAS_BASE_ADDRESS>/mgmtport_b/rw
--cred-type upwpair --cred-username manage --cred-password '!manage'
hpsp_credmgmt --update-cred -s
couplet:couplet/array/<MSA_SAS_BASE_ADDRESS>/mgmtport_b/ro
--cred-type upwpair --cred-username monitor --cred-password
'!monitor'

Workarounds 17
 3. To verify your installation of the credentials, list the credentials by entering the following
command:
hpsp_credmgmt --get-cred -s couplet:couplet/* --output-sl
The command displays the credentials for your system. Your output would differ from the
following example to reflect your system configuration:
10.125.2.221|monitor|!monitor|couplet:couplet/array/500c0ff1390c0000/mgmtport_b/ro
10.125.2.221|manage|!manage|couplet:couplet/array/500c0ff1390c0000/mgmtport_b/rw
10.125.2.220|monitor|!monitor|couplet:couplet/array/500c0ff1390c0000/mgmtport_a/ro
10.125.2.220|manage|!manage|couplet:couplet/array/500c0ff1390c0000/mgmtport_a/rw

Hardware monitoring will continue to show a storage cluster in a DEGRADED state when an MSA
snapshot LUN is removed due to the reserve LUN getting full or the user removing the block
snapshot from a StoreAll 9320 Storage platform. To resolve this issue, stop the monitoring process
via the /etc/init.d/hpspmon stop command. The monitoring process will be restarted
automatically and will reset the statuses.
Issues with just the StoreAll 9320 Storage Platform:
If an MSA I/O module is MISSING the status of its sensors (Capacitor, Temp) will appear
as UNKNOWN. After the I/O module is replaced there is a period of one minute during
which status for all sensors in the array is unavailable. The statuses will be corrected in the
next monitoring polling cycle.
If there is a hardware event involving a fan in a D2700 drive enclosure then the location of
the fan will incorrectly state that it is part of the enclosures power supply. The fan is a separate
FRU in the D2700 drive enclosure.
Disconnected SAS cables between D2700 small form factor drive enclosures are not detected
by the hardware. Monitoring software will not report lack of redundancy in connectivity to
drive enclosures that are connected behind the first drive enclosure. If the customer restarts
the other storage controller that has access to the enclosure or removes the other cable to the
enclosure loss of access to storage will occur.
Spurious failure indications and potential system crashes can occur if storage controller or
hard drive firmware is flashed while monitoring is active. Run the hpspmonControl
--pause command on both servers in the couplet before performing the flash operation.
Then, run the hpspmonControl --resume command on both servers in the couplet after
the flash operation is complete.
Disconnecting the cable from the HBA to an MSA or rebooting one or both of the MSA
storage controllers while hardware monitoring is active can result in unexpected hardware
events.
Port monitoring of the HP SC08Ge HBA (3GB) is not functional when the HP SNMP agents
are not running.
On an IBRIX X9720 or StoreAll 9730 Storage Platform, when a blade is reseated in the C7000
chassis, the On-Board Administrator (OA) might allow the blade to be manually powered on

18 Workarounds
 before Virtual Connect (VC) profile is applied. The server might come up with no networking and
without any error in OA. The VC sometimes complains.
The Active Fusion Manager (FM) might lose track of actively running tasks if the FM on the server
where the specific task coordinator is running is restarted. This situation can happen if the server
has failed over, restarted or if the Fusion Manager itself on that server has restarted.
The following tasks are impacted by this issue:
OnlineQuotacheck

Migrator

SnapReclamation

DataValidation

CaseInsensitive
The following symptoms might be observed:
The task may appear to be hung in the Fusion Manager task display output because it will
only display the last status it received.
Another task of the same type may fail to start on the same file system due to leftover rogue
processes from the previous task.
To resolve this issue:
1. Enter the following command:
ibrix_task k n <taskid> -F
2. Manually kill the specific coordinator process related to rogue task.
See the following table for information on how to kill each task.

Hung Task Enter the following command on one line to kill the specific coordinator process

OnlineQuotacheck /usr/local/ibrix//bin/ibrix_online_quotacheck, -f, -ugt, -M,

192.168.56.66, /ifs2, -j

Migrator /usr/local/ibrix/bin/ibhsm_analyzer --serviceid=2307 --port=9173 -D

-y -f ifs2 -L /usr/local/ibrix//log/ -j Migrator_58-ifs2
nl

NOTE: If there is a worker on each server that owns a segment of the file system, enter the
following command for each worker.
/usr/local/ibrix/bin/migrator_daemon --serviceid=4100 --port=9174 -f
ifs2 --logflag=6 -L /usr/local/ibrix//log/ -j Migrator_58-ifs2-worker1

SnapReclamation /usr/local/ibrix/bin/purge_coordinator --serviceid=5379 --port=9173

-D -y -f ifs2 -L /usr/local/ibrix//log/ -j SnapReclamation_59-ifs2

DataValidation /usr/local/ibrix/bin/validation_coord --serviceid=6147

--port=9173/usr/local/ibrix/bin/validator --jobid=60 --fsname=ifs1
--path=/ifs1 --scantype=1

CaseInsensitive /usr/local/ibrix/bin/dir_tree_case_insensitive -p -j 51

3. Restart the CUD by entering the following command:

/usr/local/ibrix/bin/ibrix_cud restart

Workarounds 19
 SMB
SMB and StoreAll Windows clients cannot be used together because of incompatible AD user to
UID mapping. You can use either SMB or StoreAll Windows clients, but not both at the same time
on the cluster.
The HPSMB server does not support connections from Linux SMB clients. The workaround is to
use NFS for Linux.
SMB DCN is supported on Windows File Explorer with the following exception:
The recursive notification flag in the notify request is ignored. Notifications are delivered only
for one directory level.
If SMB is joined to a domain before introducing LDAP ID mapping or changing the domain or
the provision mode, you should clear the name cache on each node. This step prevents unexpected
results such as the corruption of a name cache entry when the new configuration is in place. To
clear the cache, run the following command:
/opt/likewise/bin/lw-lsa ad-cache --delete-all
When the Microsoft Windows Share Management interface is used to add a SMB share, the
share path must include the StoreAll file system name. The Browse button on the MMC cannot be
used to locate the file system. Instead, enter the entire path, such as C:\data\.
The StoreAll management console GUI and CLI allow only 9000 file systems and directories to
be exported as SMB shares. However, the Microsoft Windows Share Management interface
allows you to create an SMB share that is not on a StoreAll file system. Although the share will
be available from the file serving node to which Windows Share Management was connected,
it will not be propagated to the other file serving nodes in the cluster.
The ibrix_localusers -i <user information> command fails if the user information
includes commas. To enter commas in the user information, use the management console GUI
instead of the CLI.
When you use the Windows security tab to add local users or groups to a security ACL on an
SMB file (for either file or share-level permissions), you typically specify the user to add as either
a DOMAIN\username or a MACHINE\username. On StoreAll systems, local users are displayed
as LOCAL\username, and it may seem like you should specify LOCAL\username in the Add
dialog box in Windows. However, in this case, the Windows client cannot interpret LOCAL.
Instead, specify the machine name of the server. For example, to add LOCAL\user1 to an ACL
on an SMB file shared out by serverX, specify serverX\user1 in the Add dialog box on the
security tab. If you later use the Windows security tab to look at this ACL, the server name will
have been replaced by LOCAL (the SMB server performs this remapping to ensure that local users
are symmetric between all servers in the cluster, and are not specific to any one machine name
in the cluster.)
When joining an SMB domain, the $ character cannot be used in passwords unless it is escaped
with a slash (\) and enclosed in single quotes (' '). For example:
ibrix_auth -n IB.LAB -A john -P 'password1\$like'
If you are using a Windows Vista client and running more than a single copy of Robocopy from
that client, a hang is possible. The work around is to disable the SMB2 protocol on all file serving
nodes. Complete these steps:
1. Disable the SMB2 protocol on all nodes:
ibrix_cifsconfig -t -S SupportSmb2=0
2. Restart the SMB service on all nodes:
ibrix_server -s -t cifs -c restart

20 Workarounds
 You may also need to restart the Windows client, as the original negotiated protocol, SMB2, may
be cached by the client. Restarting the client renegotiates the protocol back to SMB1.
Be sure to remove Active Directory users from the StoreAll share admin list before removing them
from Active Directory. If you remove an Active Directory user from Active Directory before removing
the user from the StoreAll share admins list, an error is reported when you attempt to change the
share admins list. If you are seeing errors from this situation, rejoin Active Directory and remove
all share admins. For example:
ibrix_auth -n ib.lab -A administrator@ib.lab -P fusion -S "share
admins="
Then run ibrix_auth again to specify the new list of share admins:
ibrix_auth -t -S "share admins=[ib\Administrator]"
Do not use spaces in share names. The GUI cannot create a share if there is a space in the name.
SMB clients have a 45 second timeout on every SMB request they make. If a request's timer
expires before a response comes, the client disconnects and terminates the SMB connection with
this error message: The specified network name is no longer available. After
this, the client must reconnect to the SMB server. If you are seeing this message from clients
connected to a busy server, they may be encountering timeouts from overload. For more information
about SMB client timeouts, see http://support.microsoft.com/kb/102067.

LDAP
When configuring LDAP on the GUI, double quotes are not supported in any fields of the LDAP
configuration window, including the password field. Also, escaped slashes are not supported. If
you enter \\; it will be parsed literally as \\ instead of \. Also, the ibrix_ldapconfig
command supports double quotes only at the beginning and end of a value. For example:
ibrix_ldapconfig -a -h 10.30.239.174 -b "cn=Manager,dc=ldapx,dc=net"
-p "mypassword "
The command does not support escaped slashes.

LDAP ID mapping
When LDAP ID mapping is configured with the GUI or CLI, the following fields are not validated
to ensure successful communication with the LDAP server: LdapBase, LdapServer, LdapUserDn,
LdapLapMaxWaitTime, LdapMaxEntries fields. It may be helpful to perform a manual ldapsearch
test with the intended configuration values to assure success when configuring LDAP ID mapping.
The LDAP ID mapping features depends on successful communication with the configured LDAP
server.
The openldap-clients-<version> RPM is required to perform ldapsearch commands
from the command line. This RPM is available on the OpenLDAP site.
Increasing the values of the LdapMaxWaitTime and LdapMaxEntries can affect the performance
of name resolutions for SMB and non-SMB users in large LDAP directory environments. The AD
name cache protects against excessive LDAP searches. The default value for these parameters is
10. The range of values is 0 (unlimited) or 132767.

Block snapshots
Snapshot creation may fail while mounting the snapshot. The snapshot will be created successfully,
but it will not be mounted. Use the following command to mount the snapshot manually:
ibrix_mount -f <snapshotname> -m /<snapshotname>
Quotas are disabled on block level snapshots (for example, MSA2000 snapshots) and the quota
information from the origin file system is not carried to the block level snap file system. Block level
Workarounds 21
 snapshots are temporary file systems that are not writable. Users should not query quota information
against block level snap file systems.
After the initial creation of a snapshot, it can take 4 to 6 minutes to mount the snapshot.
Block snapshots are not created when a volume group is mapped to more than one logical volume.

Remote Replication
Remote replication might fail to transfer Windows Access Control Lists (ACLs) if cross-protocol
ACL synchronization is enabled on the source cluster nodes, but not on the target cluster nodes.
Before starting the replication, ensure that cross-protocol ACL synchronization is enabled on all
source and target cluster nodes.
When remote replication is running, if the target file system is unexported, the replication of data
will stop. To ensure that replication takes place, do not unexport a file system that is the target
for a replication (for example, with ibrix_crr_export -U).
Remote replication will fail if the target file system is unmounted. To ensure that replication takes
place, do not unmount the target file system.
When continuous remote replication is used and file serving nodes are configured for High
Availability, you will need to take the following steps following failure of a node:
1. Stop continuous remote replication.
2. After the migration to the surviving node is complete, restart continuous remote replication
to heal the replica.
If these steps are not taken, any changes that had not yet been replicated from the failed node
will be lost.
No alert is generated if the continuous remote replication target becomes unavailable. Confirm
the connection to the target system by issuing a ping command and by inspecting
ibrcfrworker.log.
Sparse files on the source file system are replicated unsparse on the target. That is, all blocks
corresponding to the file size are allocated on the target cluster. Consequently, if the target file
system is the same size as the source file system, remote replication can fail because there is no
space left on the target file system. To work around this situation, if the source system contains
large sparse files, be sure that the target file system is larger than the source file system, and large
enough to fit all files in an unsparsed manner.
The mountpoint /mnt/ibrix is reserved for remote replication. Hiding or blocking this mountpoint
by mounting anything over the parent /mnt will prevent Run Once replication from working at
all, and the initial domain scan of Continuous replication will fail.
When a replication task encounters a file on the target system that was added or changed outside
of a previous replication, the replication task stops and does not replicate the remaining files from
the source to the target.
If the target file system is unmounted while an inter-cluster or intra-cluster CRR task is running, the
CRR task will not work properly. Before unmounting a file system, it is a best practice to verify
that the file system is not the target for a CRR task. If the file system is the target for a CRR task,
stop the task before unmounting the file system.

22 Workarounds
Data retention and validation
The ibrix_reten_adm command fails if the date string specified with -e contains spaces. As
a workaround, use the following command to enter date strings containing spaces:
/usr/local/ibrix/sbin/ibr_reten_adm -e expire_time -f FSNAME -P PATHLIST
The ibrix_vs_snap command cannot delete a block snapshot file system that is enabled for
data retention. Instead, use the ibrix_fs command with the -R option. For example:
ibrix_fs -d -f block_snap_ifs2_1 -R
When the GUI or ibrix_reten_adm -r is used to remove a legal hold from a file, it displays
invalid errors about autocommit; however, the command completes successfully. You can ignore
the error messages.
Directories are not supported in the path list for the ibrix_reten_adm command. However,
the command does not display the correct error message when the path list includes a directory.
PDF reports do not display on Internet Explorer with Adobe Reader 10.1.2 installed. Reports
display properly with earlier versions of Adobe Reader or with other supported browsers.
The ibrix_reten_adm -e command fails for WORM files that are not retained. If a WORM
file is not retained (either it has never been retained, or it was retained but its retention period
has expired), use the touch command to set its last accessed time (atime) forward to the desired
expiration time. The file will then be retained until that time.
PDF reports cannot be generated from the GUI when using Windows Internet Explorer 8. To
display the PDF, use one of these options:
On the browser, right-click in the white border surrounding the grey area, and select View
Source. Locate the src= line in the source. For example:
src="/fusion/reporting/retention/retention_ifs1.pdf#scrollbar=0"
width='100%' height='100%'/>
Prepend https://<hostname> to the path specified in the source and enter that URL in
the browser. Using the previous example, you would enter:
https://<hostname>/fusion/reporting/retention/retention_ifs1.pdf

Log into the node specified as the hostname in the URL and go to the directory /usr/local/
ibrix/reports/output/retention, which contains the report.

Antivirus support
If McAfee is used as a virus scan engine, when virus definitions are updated on the virus scan
engine, the updated definitions are not sent automatically to the StoreAll servers. To correct this
condition, request hotfix HF692368 from McAfee.
For performance reasons, when an NDMP backup is in progress, Antivirus scans are not triggered
for the files being backed up. The StoreAll server excludes the process corresponding to NDMP
from scanning any files on a cluster with Antivirus enabled.
HP recommends that you disable Antivirus on the file system when a non-NDMP backup is in
progress. StoreAll cannot exclude a non-NDMP backup process from scanning files during a
backup. Disabling Antivirus prevents scans for files being backed up, which improves backup
performance.
When WebDAV is enabled, the internal WebDAV files in the root of the file system are scanned.
To exclude all .lock/.pag files in the .DAV and .webdav directories from scanning, complete
these steps:
1. After the WebDAV share is created, unset the immutable bits on .webdav directory:
# chattr -i .webdav/

Workarounds 23
 2. Run the following command to exclude all files in the .webdav directory:
# ibrix_avconfig -a -E -f FS1 -P /FS1/.webdav/ -x '*'
3. Reset the immutable bit on the .webdav directory:
# chattr +i .webdav/
4. Exclude all files in the .DAV directory for each HTTP/WebDAV share created:
# ibrix_avconfig -a -E -f FS1 -P /FS1/httpshare/.DAV/ -x '*'

Segment evacuation
The segment evacuator cannot evacuate segments in a READONLY, BROKEN or UNAVAILABLE
state.

The ibrix_collect command

If collection does not start after a node recovers from a system crash, check the /var/crash/
<timestamp> directory to determine whether the vmcore is complete. The command
ibrix_collect does not process incomplete vmcores. Also check /usr/local/ibrix/log/
ibrixcollect/kdumpcollect.log for any errors.
If the status of a collection is Partially_collected, typically the management console service
was not running or there was not enough space available in the /local disk partition on the
node where the collection failed. To determine the exact cause of a failure during collection, see
the following logs:
/usr/local/ibrix/log/fusionserver.log

/usr/local/ibrix/log/ibrixcollect/ibrixcollect.py.log
Email notifications do not include information about failed attempts to collect the cluster
configuration.
In some situations, ibrix_collect successfully collects information after a system crash but
fails to report a completed collection. The information is available in the /local/ibrixcollect/
archive directory on one of the file serving nodes.
The ibrix_collect command supports a maximum collection size of 4 GB. If the size of the
final collection exceeds 4 GB, the collection fails. You must either:
Delete the excess older logs from each node to reduce the size.
Or
Manually collect the individual collection, which is stored on each node in the following
format:
/local/ibrixcollect/<node_name>_<collection_name>_<time>.tgz

Cluster component states

Changes in file serving node status do not appear on the management console until 6 minutes
after an event. During this time, the node status may appear to be UP when it is actually DOWN
or UNKNOWN. Be sure to allow enough time for the management console to be updated before
verifying node status.
Generally, when a vendorstorage component is marked Stale, the component has failed
and is not responding to monitoring. However, if all components are marked Stale, this implies
a failure of the monitoring subsystem. Temporary failures of this system can cause all monitored

24 Workarounds
 components to toggle from Up, to Stale, and back to Up. Common causes of failures in the
monitoring system include:
Reboot of a file serving node

Network connectivity issues between the management console and a file serving node

Resource exhaustion on a file serving node (CPU, RAM, I/O or network bandwidth)
While network connectivity and resource exhaustion issues should be investigated, they can occur
normally due to heavy workloads. In these cases, you can reduce the frequency at which
vendorstorage components are monitored by using the following command:
ibrix_fm_tune -S -o vendorStorageHardwareStaleInterval=1800
The default value of this command is 900; the value is in seconds. A higher value reduces the
probability of all components toggling from Up to Stale and back to Up because of the conditions
listed above, but will increase the time before an actual component failure is reported.

SNMP
When the StoreAll SNMP agent is configured, do not restart snmpd. If the snmpd restart
command is run on the active node, you will need to take one of these steps:
If Phone Home was configured, reconfigure it.

If Phone Home was not configured, reconfigure the StoreAll SNMP agent.

HP Insight Remote Support

After HP Insight Remote Support is configured, do not restart snmpd. See SNMP (page 25) for
more information.
In certain cases, large number of error messages such as the following appear in /var/log/
hp-snmp-agents/cma.log:
Feb 08 13:05:54 x946s1 cmahostd[25579]: cmahostd: Can't update OS
filesys object: /ifs1 (PEER3023)
This error message occurs because the file system exceeds <n> TB. (This situation will be corrected
in a future release.) To disable logging, edit the script /opt/hp/hp-snmp-agents/server/
etc/cmahostd and remove the option -l <logname>. Then restart the agents using service
hp-snmp-agents restart.
If you want to keep logging enabled, be aware that the log messages occur frequently, and you
will need to monitor and clean up the log file regularly to avoid filling the file system.
If Fully Qualified Domain Name (FQDN) resolution is not configured properly for the hosts, the
following error appears when hpsmhd is restarted:
Could not reliably determine the server's fully qualified domain name,
using <ip> for ServerName
To configure FQDN resolution, use either the /etc/hosts file or the Domain Name Service.
If SNMP is logging excessively, add the following command to the file /etc/sysconfig/
snmpd.options to stop the logging:
OPTIONS="-LS 0-4 d -Lf /dev/null -p /var/run/snmpd.pid -a"
To enable the default SNMP logging, add a pound sign (#) to the following command in the file
/etc/sysconfig/snmpd.options and save the file:
# OPTIONS="-LS 0-4 d -Lf /dev/null -p /var/run/snmpd.pid -a"
Then restart the SNMP service:

Workarounds 25
 service snmpd restart
On 9320/X9720 systems, the virtual SAS manager and VC connect manager must be configured
manually for HP Support.
On HP SIM 7.1, 9320/9300 nodes are discovered as HP 9320 NetStor FSN if MSA storage
is registered on the cluster nodes. This does not affect sending alerts to HP Support.

ibrix_fsck command
The log files for user-initiated tasks (such as running ibrix_fsck and migrating or rebalancing
segments) are not rotated automatically. To control the space used by these log files, you will
need to maintain them manually. The log files are located in /usr/local/ibrix/log.
After running a tiering, rebalance or segment evacuation job, a phase 1 of the ibrix_fsck
command might display a message regarding bad replica inodes. The following is an example
of the message from a phase 1 ibrix_fsck log:
Entry 'F000000060000FEAF.DAT' in
/.ibrixShadowRoot/s_1/g_367/i_100b335b7_edd68d01/D00000006000005FE
(11812038) has bad replica inode 17877133.
This message indicates that a replica inode was deleted from the file system by a user application,
but the directory entry (DENTRY) pointing to the replica has not been removed. There is not an
issue with the file system, and this situation does not impact file system operations. The DENTRY
that points to the deleted replica can be removed by running a phase 1 ibrix_fsck in corrective
mode. See the HP StoreAll Storage CLI Reference Guide for more information.

Windows Client
If the StoreAll Windows Client is registered with Fusion Manager already, clicking on Start Service
After registration under the Registration tab displays a Response Code =400 error. Click the
Recover Registration option instead.
Reading of snaps as memory mapped files is not supported. Undesired results can occur if a snap
file is used as memory mapped file while the current version of the file is being modified by a
different instance.

General
The ibrix_pv -a -o mpath command does not recognize a multipath device. The command
works properly when an argument is added to ignore the standard SCSI devices. Execute the
command as follows, specifying the path to your device.
ibrix_pv -a -o accept:/dev/mapper/mpath10,~standard
The ibrix_lv -d -s <range of Segments> command sometimes displays a failed
command message. Although the segments for which the command fails are still listed by the
ibrix_lv -l command, they are successfully removed from the underlying Operating System
LVM view. This issue is a timing condition for large file system delete operations where the delete
command times out while waiting for a response from the storage. Although the deletion was
successful the command reports it as failed and StoreAll preserves the associated volume group
(ivg) and logical volume (ilv). This issue has been observed on the StoreAll 9730.
To resolve this issue, rerun the logical volume delete command using the -X override option, for
example: ibrix_lv -d -s <ilv#> X
The override option forces the removal of the StoreAll Logical Volume record from the StoreAll
database.

26 Workarounds
 When using the ibrix_vs -a command during file system extension, the command fails on a
combined cluster configuration (such as 9730 and 9320) with the following error message:
Lun already set for physical volume dxx can two vendorstorages see the
same luns?
Contact HP Support for assistance.
You might not be able to mount a file system after an unmount if you:
1. Stop the metabox.
2. Evacuate the segment.
3. Failover of a server (ibrix_server -f -h <server_name>)
4. Unmount the file system.
To resolve the issue, restart the affected server before unmounting the file system.
If the ibrix_auth command fails after a QR restore of a server in a cluster with a message
similar to:
ibrix_auth -n IBRQA1.HP.COM -A administrator@ibrqa1.hp.com -P password
-h hostnameX
Iad error on host hostnameX failed command (<HIDDEN_COMMAND>) status
(1) output: (Joining to AD Domain:IBRQA1.HP.COM With Computer DNS Name:
hostsnameX.ibrqa1.hp.com )
Verify the contents of the /etc/resolve.conf file is not empty. If it is empty, copy the contents
of the /etc/resolve.conf file on another server to the empty resove.conf file.
When configuring automated failover with the HA wizard for the X9720, you must create the
ibrixpwr iLO user on each node before the cluster HA can be fully functional. Enter the
following command on each cluster node to create an iLO user with the user name ibrixpwr
and the password hpinvent:
ibrix_ilo c u ibrixpwr p hpinvent
HP-OpenIPMI does not work with the HP-Health service. For example, Proliant health check tools
such as SMH and SIM and hpasmcli commands such as SHOW POWERSUPPLY do not report
errors. This situation occurs because StoreAll OS software requires standard RHEL IPMI. Remove
the HP-OpenIPMI RPM (execute rpm -e HP-OpenIPMI), and then start the standard RHEL IPMI
(execute /etc/init.d/ipmi start). The standard RHEL IMPI will now start automatically
when the server is booted.
During server migration or failover, certain cluster events will be reported as alerts. These events
are expected and normal, and are reported temporarily as a server is failing over to another
server.
Node failover does not occur when a node has a complete loss of power (for example, removing
power cords or pulling a blade from a chassis). Do not test high availability in this manner.
NFS locks may return an error code (37) after an unlock operation even though the lock was
correctly released. If other results are normal, you can ignore this error code.
On 9730 systems, the iLO can fail to power up a server when directed to do so from the iLO web
page or the iLO's Integrated Remote Console. To correct this condition, reset the iLO processor.
This can be done from the iLO web UI; select Reset on the Diagnostics page.
After a server is rebooted following a QR install and before the server is configured, an error
message reports that the SRVSVCD service failed to load. The failure occurs because the service
requires that the network subsystem be configured. The error message can be safely ignored.
After the network is configured, restart the SMB service.

Workarounds 27
Installation instructions
New installations
HP StoreAll OS software is preinstalled on supported devices. If you need to reinstall the software, see
the administrator guide for your storage system.

Upgrades
The upgrade procedure is provided in the administrator guide for your storage system. Contact HP
Support for assistance with the procedure.

Compatibility/Interoperability
Note the following:
Every member of the cluster must be running the same version of StoreAll OS software.
The cluster must include an even number of file serving nodes.

Documentation feedback
HP is committed to providing documentation that meets your needs. To help us improve the
documentation, send any errors, suggestions, or comments to Documentation Feedback
(docsfeedback@hp.com). Include the document title and part number, version number, or the URL when
submitting your feedback.

28 Installation instructions