SoNAS ACE

Copyright BM Corp. 2012. All rights reserved.
165
Chapter 4. Active CIoud Engine
BM Active Cloud Engine, the core of BM SONAS and BM Storwize V7000 Unified,
provides users with the ability to manage files efficiently, locate the data of interest rapidly,
and get that data to where it is needed seamlessly.
n this chapter, we provide an overview of BM Active Cloud Engine and then describe in
detail the Policy, ntegrated Lifecycle Management, and WAN caching components.
4
166 SONAS mplementation and Best Practices Guide
4.1 SONAS Active CIoud Engine: Overview
The SONAS Active Cloud Engine (ACE) is an advanced form of multiple site replication
designed by BM Research. ACE is designed to address an T world where Cloud Storage is
the goal, yet we are still faced with not just islands of storage, but also "islands of clouds.
Different types of cloud implementations cannot dynamically exchange data, and moving data
between cloud vendors and providers can be equally difficult.
The BM Active Cloud Engine capability is designed to address these issues by providing a
means to exchange data in a cloud storage environment dynamically, immediately, and in an
on-demand manner, between multiple geographically dispersed locations and even between
different cloud storage providers. ACE does it by extending the SONAS capability for a
centrally auto-managed single highly scalable high performance namespace, to a truly
distributed world wide, geographically dispersed global namespace.
n Figure 4-1, the SONAS users see the appearance of a single global NAS storage
namespace, even though the namespace is actually physically distributed among multiple
geographically dispersed locations. This common global namespace is achieved by the
SONAS Active Cloud Engine enabled systems working together.
Figure 4-1 Manage SONAS storage centrally and automatically on global basis
The number of SONAS Active Cloud Engine sites is not fixed or limited; you can assemble as
many SONAS Active Cloud Engine sites together as you want. SONAS Active Cloud Engine
achieves a common global geo-graphically dispersed namespace and enables efficient
multi-site global file management, as follows:
Automatically caching files at remote sites, providing local latency reduction, higher
availability, while reducing bandwidth costs
SONAS Active Cloud Engine
L
o
g
i
c
a
l
P
h
y
s
i
c
a
l
Global policy engines
acting in concert
1) Remove Latency
2) Geo-distribute data
SONAS
file servers
Protocol
s
CIFS
NFS
HTTP
FTP
SCP
Manage
ment
Central
Administr
ation
Monitorin
g
File Mgmt
Availabili
ty
Data
Migration
Replicatio
n
Backup
SONAS
policy engine
Protocols
CIFS
NFS
HTTP
FTP
SCP
Management
Central
Administration
Monitoring
File Mgmt
Availability
Data Migration
Replication
Backup
IBM SONAS
policy engine
Protocol
s
CIFS
NFS
HTTP
FTP
SCP
Manage
ment
Central
Administr
ation
Monitorin
g
File Mgmt
Availabili
ty
Data
Migration
Replicatio
n
Backup
Protocol
s
CIFS
NFS
HTTP
FTP
SCP
Manage
ment
Central
Administr
ation
Monitorin
g
File Mgmt
Availabili
ty
Data
Migration
Replicatio
n
Backup
SONAS
policy engine
SONAS policy engine
Geo-disperse
storage
Appears to users as one
SONAS global namespace
Network
Chapter 4. Active Cloud Engine 167
Providing automatic pre-fetching of files, that is, on-demand pull
Providing capability to push files and/or updates to either the remote sites, or back to the
home sites, according to policy
Virtualizing all of the Active Cloud Engine sites to a single view, all of the physically
separated SONAS, GPFS, or V7000 Unified system, thus providing a single global
namespace
The power of the SONAS Active Cloud Engine can be understood by examining its usage
when applied to a worldwide business storage cloud, as shown in Figure 4-2.
Figure 4-2 SONAS Active Cloud Engine deployed in a worldwide business storage cloud
The Active Cloud Engine appears to be a standard NFS file server in each site; any NFS
clients, proxies, or users in that site can access the Active Cloud Engine storage through
NFS.
More importantly, each SONAS Active Cloud Engine also acts as a gateway to all other
Active Cloud Engine sites, presenting a common view and common global namespace of all
other sites in the cloud storage. With proper authority, all users at an individual site can see
the entire single global cloud storage namespace across all sites.
Among this cloud storage Active Cloud Engine collection, files are automatically, on-demand
moved and cached at the Active Cloud Engine in the local site. The central locations can also
request data ingested at any of the remote sites, or data can be pushed from any site to any
other sites.
nteroperability is assured because each site sees the BM Active Cloud Engine gateway as a
standard NFS server. Therefore, other NAS storage, proxies, or NAS clients at any site can
be of any vendor or cloud storage deployment that supports standard NFS.
SONAS Active Cloud Engine:
applied to a worldwide business storage cloud
Metro City 2
Metro City1
Acti ve Clou d Engi ne
Central ap pli ance
Proxy
Proxy
Proxy
Proxy
Proxy
Acti ve Clou d Engine
Central Ap pli ance
Proxy
Proxy
Proxy
Proxy
Proxy
Active Clo ud Engin e
Cach e
Proxy
Proxy
Proxy
Proxy
Proxy
Acti ve Clo ud Eng ine
Cach e
Proxy
Proxy
Proxy
Proxy
Proxy
Active Clo ud Engin e
Cache
Proxy
Proxy
Proxy
Proxy
Proxy
Acti ve Clo ud Eng ine
Cach e
Proxy
Proxy
Proxy
Proxy
Proxy
Active Cl oud En gi ne
Cache
Proxy
Proxy
Proxy
Proxy
Proxy
R
e
p
lic
a
tion
SONAS Panache
Cache
Regional City
Proxy
Proxy
Regional City
Regional City
Regional City
Regional City
Regional City
IBM SONAS as a media storage platform with a single name space - globally
Notes:
Media / Movie Files injected i n
Metro Ci ty 1 and i s repli cated to
Metro Ci ty 2 for DR fai lov er .
Remote SONAS appli ances wi ll
show the fil e ex ists and wi ll
upl oad on fi rst acc ess to local
store
Content Deli very platform
software runs and c oordinates
acc ess to media fi les and li ve
streams vi a proxy nodes.
End dev ices (Smart Phones,
iPads, Tablets , IP enabl ed
Streaming Devi ces etc) wil l
downl oad fil es on demand .
Prox y serv ers use the SONAS
Acti ve Cloud Engi ne appl iance
as their l ocal storage
SONAS Ac tive Cl oud Engine
Appli ances are siz ed to match
performanc e and capac ity
demands at cache sites. Ac ti ve
Cl oud Engine will automati cal ly
cac he fi les to the avai lable
cac he s ize.
Example:
SONAS Active Cloud Engine acts as a
Storage Cloud Gateway in each geographi c location
By fusing standard NFS protocols with BM SONAS central policy managed storage
capabilities, BM Active Cloud Engine bridges not only multiple geo-dispersed BM SONAS
sites, but also provides for dynamic, automatic, policy-managed cloud storage data
movement between multiple different cloud storage implementations or vendors at each site.
BM Active Cloud Engine thus provides a unique high-performance capability to auto-align
geographically dispersed or local cloud storage, automatically managed and synchronized
across all sites using BM Active Cloud Engine capabilities. BM Active Cloud Engine provides
the ability to make a multi-site and multi-company cloud storage, an active reality.
Next we look at how SONAS Active Cloud Engine implements these capabilities.
4.1.1 SONAS Active CIoud Engine: GIobaI, in more detaiI
The SONAS Active Cloud Engine (ACE) provides the concept of "home clusters and "cache
clusters. Cache clusters act as front end, wide area network cache access points, which can
transparently access the entire collection of Active Cloud Engine SONAS systems.
Active Cloud Engine consists of a client-server architecture:
The home cluster (server) provides the primary storage of data.
The cache cluster (clients) can read or write cache data that is exported to them.
These concepts are shown in Figure 4-3.
Figure 4-3 SONAS Active Cloud Engine basics
acting in concert
Geo-dispersed
storage
SONAS
Interface
node
Interface
node
Interface
node
Active Cl oud Engi ne
Home cluster
Policy Engine
Home cluster provides the
primary storage of data
Cache clusters will
read or write cache
data that is
exported to them
Storage
nodes
Storage
nodes
Storage
nodes
Storage
nodes
Int erf ace
node
Active Cloud Engine cache cluster
Policy Engine
Storage
node
Interface
node
Storage
node
Interface
node
Policy Engine
Storage
node
Interface
node
Storage
node
I nterface
node
Policy Engine
Storage
node
Interface
node
Storage
node
I nterface
node
Network
n the Active Cloud Engine, we have the following terminology:
Home. The home cluster exports a fileset by standard NFS protocol, over one or more
nterface nodes, as defined by policies:
- Users are verified as having permission to access the file from the home cluster
- A home is any NFS mount point in the file system. Home designates which site is the
owner of the data in a cache relationship. Note that the writing of the data might be at
another, different location in the ACE configuration. However, the home cluster is the
owner of the relationships.
- The NFS mount point can be any SONAS file system or fileset within a filesystem. At
any one point in time, there is one home location. You can have an unlimited number of
cache relationships, and home locations can be changed or moved in coordination with
the cache relationships.
Cache. A cache cluster is a SONAS, GPFS, or V7000 Unified fileset. The cache cluster
does the majority of the work in an BM Active Cloud Engine environment:
- nterface nodes in the cache cluster communicate with the home clusters.
- Cache cluster nterface nodes communicate with the home clusters using standard
NFS.
- The cache cluster presents the virtualized image of all authorized Active Cloud Engine
namespaces to the local user as a single virtualized global namespace view.
These concepts are illustrated in Figure 4-4.
Figure 4-4 SONAS Active Cloud Engine - home cluster and cache cluster
/home/appl/data/web/spreadsheet.xls
/home/appl/data/web/drawing.ppt
IBM Active Cloud Engine
/home
/appl
/data
/web
/home/appl/data/web/drawing.ppt
/home/appl/data/web/spreadsheet.xls
N
F
S
e
x
p
o
rt
to
c
a
c
h
e
c
lu
s
te
r
Cache cluster presents image
of all Active Cloud Engine
clusters to local users
Read: If data not in cache
cluster, data requested from
home
Cache cluster inter face node(s)
communicates with home cluster(s)
M
u
ltip
le
p
a
ra
lle
l
s
tre
a
m
s
Policy Engine
Storage
node
Interface
node
Storage
node
Interface
node
SONAS
Interface
node
Interface
node
Interface
node
Active Cloud Engi ne
Home cluster
Policy Engine
Storage
nodes
Storage
nodes
Storage
nodes
Storage
nodes
Interface
node
On a read request at the local cache, existing file data at home is pulled into the cache on
demand. Multiple nterface nodes and multiple NFS connections are used to provide high
performance.
f the cache is in write mode (for any particular fileset, a single writer across the global
namespace is supported), new data written to the local cache. This data is asynchronously
pushed back to home, while still maintaining a local copy in cache.
As of SONAS 1.3 Active Cloud Engine, there are three modes allowed for an individual Active
Cloud Engine cache cluster fileset. Different filesets can have different modes:
Read: Cache cluster can read data, but no write data changes are allowed. Useful for data
broadcast, for latency and bandwidth reduction, for localizing data to the local users.
Single Writer: The cache cluster is defined as the only writing location in the cloud storage
global namespace. The home cluster receives the writes, does not change the data, the
home cluster acts as central repository so that multiple other cache clusters can read the
data. Useful for remote ingest of data, transmission to home, and then reflecting that data
to all other authorized points in the cloud storage
Local Update: Data is read or pushed from home and cached at the local cache cluster.
Changes are allowed locally, but changes are not pushed back to home. After data is
locally changed, the home cluster and cache cluster relationship is marked to indicate that
cache and home are no longer in sync for this file.
Read Only and Single Writer caches can be changed (at granularity of the fileset) to any other
mode, with appropriate coordination within the Active Cloud Engine environment.
The fundamental BM Active Cloud Engine implementation requirement is that a unique
high-level directory naming schema is in place, thus allowing all Active Cloud Engine sites to
identify each ACE site by a unique high-level directory tree name.
An example of this naming schema is shown in Figure 4-5.
Figure 4-5 SONAS Active Cloud Engine - sample directory structure implementation
After the unique directory name is identified, it becomes the mount point for all of the other
SONAS Active Cloud Engine sites. As each remote cluster is mounted, the cache cluster
mode (read-only, single writer, or local update) is specified and validated. After being
validated and mounted, from that point on, all of the BM Active Cloud Engine functions work
as previously described in this chapter.
By intent, the BM Active Cloud Engine, by using standard directory tree structures as the
unique identifiers for each ACE site, minimizes integration effort and maximizes time to
implementation and time to value.
4.1.2 SONAS gIobaI Active CIoud Engine: Summary
BM SONAS architectural capabilities for scale out parallelism, high performance, and local
LM/HSM central storage policy management were extended to multiple geographic locations
and to cloud storage with the global Active Cloud Engine capabilities.
The global Active Cloud Engine is very flexible in capability and configuration, high
performance in nature, and provides a significant new technology capability for centrally
managing and deploying geographically dispersed cloud storage. Cascading is allowed and
encouraged.
File System: store1
File System: store2
Cache Filesets:
/data1
/data2
Local Filesets:
/data3
/data4
Cache Filesets:
/data5
/data6 Local Fileset s:
/ data1
/ data2
Cache Filesets:
/ data3
/ data4
Cache Filesets:
/ data5
/ data6
File System: store2
Cache Fileset s:
/data1
/data2
Cache Fileset s:
/data3
/data4
Local Filesets:
/data5
/data6
SONAS2.ibm.com
SONAS1.ibm.com
SONAS3.ibm.com
Clients connect to:
SONAS:/data1
SONAS:/data2
SONAS:/data3
SONAS:/data4
SONAS:/data5
SONAS:/data6
Clients connect to:
SONAS:/data1
SONAS:/data2
SONAS:/data3
SONAS:/data4
SONAS:/data5
SONAS:/data6
Clients connect to:
SONAS:/data1
SONAS:/data2
SONAS:/data3
SONAS:/data4
SONAS:/data5
SONAS:/data6
Each cache site sees the entire
global namespace view
Home for
data3 and
data4
Home for
dat a5 and
dat a6
Home for
dat a1 and
dat a2
IBM Active Cloud Engine
global namespace implementation
Every file set is accessible
fromall si tes
Each cache site sees
the entire gl obal
namespace view
An example of an advanced BM Active Cloud Engine configuration is shown in Figure 4-6, to
illustrate the kind of flexibility that is possible.
Figure 4-6 SONAS Active Cloud Engine configuration flexibility
The use of NFS as the standard interface and mount point provides interoperability, extending
the Active Cloud Engine capability beyond BM SONAS, BM GPFS, and BM V7000 Unified
to any NFS-capable storage, clients, and cloud storage implementation.
More usage cases for global BM Active Cloud Engine are described in the companion book
SONAS Concepts, Architecture, and Planning Guide, SG24-7963.
The creative applications for SONAS Active Cloud Engine in today's geographically
dispersed, multiple-organization cloud storage environments is only limited by our business
imagination. BM Active Cloud Engine brings a new paradigm of possibilities to cloud storage.
4.2 SONAS poIicy management
n this section, we provide information about how you can create and use SONAS policies.
We describe the following topics:
Policies
Policy rules
Policy command line syntax
Creating and managing policies
Policy rules and best practices
Sample policy creation walkthrough
Policy best practices
Policy Engine
Storage
node
Interface
node
Storage
node
Interface
node
acting in concert
Geo-dispersed storage
Central policy management
NFS
NFS
NFS
NFS
NFS
etc.
NFS
Example of
cascading
configurations
SONAS
Interface
node
Interface
node
Interface
node
Policy Engine
Storage
nodes
Storage
nodes
Storage
nodes
Storage
nodes
Interface
node
SONAS
Interfac
e node
Interfac
e node
Interfac
e node
Policy Engine
Storage
nodes
Storage
nodes
Storage
nodes
Storage
nodes
Interfac
e node
Active Cloud Engine
Home on different fileset
Policy Engine
Storage
node
Interface
node
Storage
node
Interface
node
Policy Engine
Storage
node
Interface
node
Storage
node
Interface
node
Active Cloud Engine
Home on fileset level
Network
NFS
4.2.1 PoIicies
SONAS provides a means to automate the management of files using policies and rules.
Properly managing your files allows you to efficiently use and balance your premium and less
expensive storage resources.
GPFS supports these policies:
File placement policies are used to automatically place newly created files in a specific file
system.
File management policies are used to manage files during their lifecycle by moving them
to another file system pool, copying them to archival storage, changing their replication
status, or deleting them.
A policy is a set of rules that describes the life cycle of user data based on the file's attributes.
Each rule defines an operation or definition, such as migrate to a pool and replicate the file.
There are three uses for rules:
nitial file placement
File management
Restoring file data
PIacement poIicy
When a file is created or restored, the placement policy determines the location of the file's
data and assigns the file to a file system pool. All data written to that file is placed in the
assigned file system pool.
The placement policy defining the initial placement of newly created files and the rules for
placement of restored data must be installed into SONAS. f a SONAS system does not have
a placement policy installed, all the data is stored into the system pool. Only one placement
policy can be installed at a time. f you switch from one placement policy to another, or make
changes to a placement policy, that action has no effect on existing files. However, newly
created files are always placed according to the currently installed placement policy.
Management poIicy
The management policy determines file management operations such as migration and
deletion. You can define the file management rules and install them in the file system together
with the placement rules. n either case, policy rules for placement or migration can be
intermixed. Over the life of the file, data can be migrated to a different pool any number of
times, and files can be deleted or restored. File management rules can also be used to
control the space utilization of online file system pools. When the utilization for an online pool
exceeds the specified high threshold value, SONAS can be configured to trigger an event that
can automatically start a policy and reduce the utilization of the pool.
Error checking for fiIe-pIacement poIicies
SONAS performs error checking for file-placement policies in the following phases:
When you install a new policy, GPFS checks the basic syntax of all the rules in the policy.
GPFS also checks all references to file system pools. f a rule in the policy refers to a file
system pool that does not exist, the policy is not installed and an error is returned.
When a new file is created, the rules in the active policy are evaluated in order. f an error
is detected, skips all subsequent rules, and returns an ENVAL error code to the
application.
Otherwise, the first applicable rule is used to store the file data.
4.2.2 PoIicy ruIes
A policy rule is an SQL-like statement as shown in Appendix A, "Policy rule syntax definitions
on page 581 that tells SONAS system what to do with the data for a file in a specific pool if the
file meets specific criteria. A rule can apply to any file being created or only to files being
created within a specific fileset or group of filesets.
Rules specify conditions that, when true, cause the rule to be applied. Here we list some of
these conditions:
Date and time when the rule is evaluated, that is, the current date and time
Date and time when the file was last accessed
Date and time when the file was last modified
Fileset name
File name or extension
File size
User D and group D
GPFS evaluates policy rules in order, from first to last, as they appear in the policy. The first
rule that matches determines what is to be done with that file. For example, when a client
creates a file, GPFS scans the list of rules in the active file-placement policy to determine
which rule applies to the file. When a rule applies to the file, GPFS stops processing the rules
and assigns the file to the appropriate pool. f no rule applies, an ENVAL error code is
returned. See Figure 4-7.
Figure 4-7 Sample policy syntax constructs
PoIicy ruIe types
There are eight types of policy rules that allow you to define specific actions that GPFS can
implement on the file data. Each rule has clauses that control candidate selection, namely
when the rule is allowed to match a file, what files it matches, the order to operate on the
matching files and additional attributes to show for each candidate file. Different clauses are
permitted on different rules based upon the semantics of the rule.
DefauIt fiIe-pIacement poIicy: When a file system is first created, the default
file-placement policy is to assign all files to the system storage pool.
define(stub_size,0)
define(is_premigrated,(MISC_ATTRIBUTES LIKE '%M%' AND KB_ALLOCATED > stub_size))
define(is_migrated,(MISC_ATTRIBUTES LIKE '%M%' AND KB_ALLOCATED == stub_size))
define(access_age,(DAYS(CURRENT_TIMESTAMP) - DAYS(ACCESS_TIME)))
define(mb_allocated,(INTEGER(KB_ALLOCATED / 1024)))
define(exclude_list,(PATH_NAME LIKE '%/.SpaceMan/%' OR
NAME LIKE '%dsmerror.log%' OR PATH_NAME LIKE '%/.ctdb/%'))
define(weight_expression,(CASE WHEN access_age < 1 THEN 0
WHEN mb_allocated < 1 THEN access_age
WHEN is_premigrated THEN mb_allocated * access_age * 10
ELSE mb_allocated * access_age
END))
RULE 'hsmexternalpool' EXTERNAL POOL 'hsm' EXEC 'HSMEXEC'
RULE 'hsmcandidatesList' EXTERNAL POOL 'candidatesList' EXEC 'HSMLIST'
RULE 'systemtotape' MIGRATE
FROM POOL 'silver' THRESHOLD(80,70)
WEIGHT(weight_expression) TO POOL 'hsm'
WHERE NOT (exclude_list) AND NOT (is_migrated)
RULE 'default' set pool 'system'
Add a default
Placement rule
Keep these
rules/defines
Keep this
clause
Tweak these
thresholds
Modify this
Weight
Expression
Here we explain these rules and their respective syntax diagrams:
1. File placement rule:
File placement rules specify which file system and allocation are used upon file creation.
Changing the rules that apply to a file's placement does not cause the file to be moved.

--
--
Example 4-1 creates a rule named datfiles. The newly created .dat files are saved to storage
pool named poolfordatfiles.
Example 4-1 File placement rule example
- -
2. File migration rule:
File migration rules are allowing the policy manager to coordinate file migrations from one file
system to another file system pool or to external pools.
--

--

--
--
---
--
n Example 4-2, there is no clause, so regardless of their current storage pool
placement, all files from the named filesets are subject to migration to storage pool pool2.
Example 4-2 Sample migration rule syntax
-
3. File deletion rule:
A file that matches this rule becomes a candidate for deletion.
--

--
--
--
---
--
The rule in Example 4-3 creates a rule named mpg. All files have the .mpg extension, and file
sizes that are larger than 20123456 are deleted from the system.
Example 4-3 Sample DELETE rule syntax

4. File exclusion rule:
A file that matches this rule is excluded from further rule evaluation. When specified in a LIST
rule, EXCLUDE indicates that any matching files be excluded from the list.
--

--
--
This example rule in Example 4-4 named Xsuper excludes all .mpg files that belong to
USERD 200 from deletion.
Example 4-4 Sample EXCLUDE rule syntax
-

5. File list rule:
dentifies a file list generation rule. A particular file might match more than one list rule, but
are included in a particular list only once. - provides the binding to an
rule that specifies the executable program to use when processing the generated list.
--
-

--
--
--
---
--
Example 4-5 shows how to exclude files containing the word test from the LST rule named
allfiles.
Example 4-5 Sample EXTERNAL RULE syntax
- -
- - -
Tip: Specify the EXCLUDE rule before rules that might match the file that is being
excluded.You cannot define a list and what to exclude from the list in a single rule. You
must define two LIST statements, one specifying which files are in the list, and one
specifying what to exclude from the list.
Rules must adhere to a specific syntax as documented in the Appendix A, Policy rule syntax
definitions on page 581. This syntax is similar to the SQL language because it contains
statements such as WHEN (1imeBooleanExpression) and WHERE SqlExpression. Rules
also contain SQL expression clauses that allow you to reference various file attributes as SQL
variables and combine them with SQL functions and operators. Depending on the clause, an
SQL expression must evaluate to either true or false, a numeric value, or a character string.
Not all file attributes are available to all rules.
Macro defines
Policies can be coded using defines, also called macro defines. These are essentially named
variables used to make rules easier to read. For example, the statement creates a define
named mb_allocated and sets it to the size of the file in MB.

Defines offer a convenient way to encapsulate weight expressions so as to provide common
definitions across the policy. These common exclusions are typical:
The "special file migration exclusion definition: Always use it when migrating.
The "migrated file migration exclusion definition: Always use it when migrating.
4.2.3 Semantics of the runpoIicy command and its poIicies
File management policies are executed and evaluated by the command.
A file can be a potential candidate for only one migration or deletion operation during one
run; only one action is performed. The SONAS command uses the
SONAS can engine to determine the files on which to apply specific actions. The SONAS
scan engine is based on the GPFS command in the background, and
runs in three phases.
Phase one
Phase one selects candidate files. All files in the selected filesystem device are scanned and
all policy rules are evaluated in order for each file. Files are either excluded or made
candidates for migration or deletion, and each candidate file is assigned a weight or priority.
Thresholds are also determined and all the candidate files are sent as input to the next phase.
The attributes of each file are read from the file's GPFS inode structure.
Policy rules in order
For each file, the policy rules are considered, in order, from first rule to last:
f the rule has a clause that evaluates to , the rule is skipped.
f the rule has a clause, and the named pool does not match the
attribute of the file, the rule is skipped.
f the clause is satisfied, but there is also a clause, and if the
occupancy percentage of the named pool is less than the HighPercentage parameter of
the clause, the rule is skipped.
f the rule has a clause, but none of the named filesets match the
attribute of the file, the rule is skipped.
f the rule has a WHERE clause that evaluates to , the rule is skipped. Otherwise, the
rule applies.
f the applicable rule is an rule, the file is neither migrated not deleted. Files
matching the rule are not candidates for any or rule.
EXCLUDE clause from LIST rule
To exclude files from matching a rule, you must create a separate rule with the
clause and place it before the rule.
f the applicable rule is a rule, the file becomes a candidate for migration to the
pool specified by the clause.
f the applicable rule is a rule, the file becomes a candidate for deletion.
f there is no applicable rule, the file is not a candidate for migration or deletion.
Each candidate file (for migration or deletion) is also associated with a LowPercentage
occupancy percentage value, which is taken from the clause of the applicable
rule. f not specified, the value defaults to 0%.
Each candidate file is also associated with a numeric weight, either computed from the
-- of the applicable rule, or assigned a default using these rules:
- f a is specified within a clause of the applicable rule, the
weight of the candidate is taken as the attribute of the candidate file.
- f a is not specified within a clause of the applicable rule, the
weight of the candidate is taken as .
Phase two
Phase two chooses and schedules files. t takes the output of phase one and orders it so that
candidates with higher weights are chosen before those with lower weights. Files are grouped
into batches for processing, typically according to weight, and the process is repeated until
threshold objectives are met or until the file list is finished. Generally, files are not chosen in
this phase after the occupancy level of the source pool falls below the low threshold or when
the occupancy of the target pool is above the limit or 99% of total capacity. Generally,
candidates with higher weights are chosen ahead of those with lower weights.
File grouping and the SIZE clause
When scheduling files, mmapplypolicy simply groups together the next 100 files by default.
However, you can set up the policy to schedule files so that each invocation of the HSM
nterface Script gets approximately the same amount of file data to process. To do so, use the
clause of certain policy rules to specify that scheduling be based on the sum of the sizes
of the files. The clause can be applied to the following rules. For details on policy rules
syntax, see Appendix A, "Policy rule syntax definitions on page 581.
DELETE
EXTERNAL LST
EXTERNAL POOL
LST
MGRATE
As a best practice, specify the rule before any other rules that might match the
files that are being excluded. For example:
-
- -
Migrates all the files that are not owned by root. f the rule is placed in the policy
file before the rule, all files are migrated, because the policy engine evaluates the
rules from first to last, and root's files must match the rule.
Reasons for candidates not to be chosen for deletion or migration
Generally, a candidate is not chosen for deletion from a pool, nor migration out of a pool,
when the pool occupancy percentage falls below the value. Also, candidate
files are not chosen for migration into a target when the target pool reaches the
occupancy percentage specified by the clause (or 99% if no was explicitly
specified by the applicable rule).
Phase three
Phase three performs the actual file migration and deletion. The candidate files that were
chosen and scheduled by the second phase are migrated or deleted, each according to its
applicable rule. For migrations, if the applicable rule had a clause, the replication
factors are also adjusted accordingly. t is also possible for the source and destination pools to
be the same because it can be used to adjust the replication factors of files without
necessarily moving them from one pool to another.
4.3 Creating and managing poIicies
n this section, we describe what policies and rules consist of, including examples of policies
and rules, and we describe the SONAS commands that manage policies and rules. We
illustrate how to create a file system pool and extend a filesystem to use the file system pool.
We then show how to create and apply data allocation policies.
4.3.1 FiIe poIicy types
File placement policies for a filesystem are set using the - command evaluated
when a file is created. f no file placement rule is in place, GPFS stores data on the system
pool, also called system.
4.3.2 FiIe poIicy use
File management policies are used to control the space utilization of online file system pools.
They can be tied to file attributes such as age and size and also to pool utilization thresholds.
The file management rules are evaluated periodically when the command is
executed or when a task scheduled with the - is executed.
4.4 SONAS CLI poIicy commands
The SONAS CL has multiple commands to create and manage policies. Policies are created
using the and commands.
Tip: The migration performed in the third phase can involve large amounts of data
movement. Therefore, you might want to consider performing the data movements with the
-- command.
Figure 4-8 shows the CL policy commands and their interaction.
Figure 4-8 CLI policy commands and their interaction
4.4.1 mkpoIicy command
The command creates a new policy template with a name and a list of one or more
rules. The policy and rules are stored in the SONAS management database and a validation
of the rules is not performed at this time. The command is invoked as follows:
-
The policy has a name and a set of rules specified with the switch. The switch sets the
default policy for a filesystem. Optionally a policy can be created by copying an existing policy
or a predefined policy template with the command and the option.
The policy is later applied to a SONAS filesystem.
The rules for a policy must be entered as a single string and separated by semicolons and
there must be no leading or trailing blanks surrounding the semicolons.
t can be accomplished one of two various ways:
The first method is to enter the rule as a single long string as shown in Example 4-6.
Example 4-6 Rule entered as single string
- - -
-- --
--
The second method uses the Linux line continuation character (backslash) to enter rules
as shown in Example 4-7.
Example 4-7 Rule entered using continuation character
-
- -
SONAS cluster
SONAS db
filesys22
policy1
lspolicy
list policies
1-all defined in db or
2-specific policy details
3-applied to all filesys
filesys44
policy1
cron
Filesys44
when to run default
applied filesys policy?
mkpolicy
create policy
chpolicy
change policy
rmpolicy
remove policy
setpolicy
apply policy to fs
for new files
runpolicy
execute policy on fs
for existing files
policy1
rule1
rule2
policy7
rule3
rule4
mkpolicytask
schedule policy
rmpolicytask
remove
schedule policy
-- --
--
n Example 4-8 we show sample uses of the command:
Create a policy with the name "test with two rules assigned:
- - --
Create a policy with the name "test_copy as a copy of the existing policy "test:
- -
Create a policy with the name "default with two rules assigned and marks it as the default
policy:
- --
Example 4-8 Sample mkpolicy command
- - - -

---
4.4.2 Changing poIicies using the chpoIicy command
The command modifies an existing policy by adding, appending or deleting rules
and the can remove a policy from the SONAS database but it does not remove a
policy from a filesystem.
To change a policy, submit the command, specifying the policy name and either the
--add or --remove option. For example, to change a policy named hsmpolicy to remove the
current systemtotape rule and add a new systemtotape rule, submit the following commands
shown in Example 4-9.
Example 4-9 Sample chpolicy command
- --
- -- --
-- - -
-
You can also add a rule and insert it before another rule by using the --before option. For
example, to include rule named PoolXtotape before rule named systemtotape, submit the
command shown in Example 4-10.
Example 4-10 Sample chpolicy command using the --before parameter
-
- - --
Example 4-11 is an example of the command with the option.
Example 4-11 Sample chpolicy command using the --add parameter
- - --
---
- - -
-
- - - -

- --
---
The command allows you to check policy syntax and to test the policy as shown in
Example 4-12.
Example 4-12 Sample chpolicy command testing policy syntax
- - -
-
Here, <> specifies the filesystem and the policy contained in the
database to be tested. Without the option, the policy is only checked for correctness
against the file system. Using the option does a test run of the policy, outputting the result
of applying the policy to the file system and showing which files are migrated, <-
indicates if the chkpolicy command is to be executed on the specified list of nodes of the
cluster where this option is used. f no cluster nodes are specified, the chkpolicy command is
executed either on an arbitrary node from the list of HSM enabled nodes or on an arbitrary
node. Use the - command to view the output from the command.
Example 4-13 is a command with the showlog command specified.
Example 4-13 Checking policies for correctness example
- - -
- - -
- -
-

--
- -
- ---
-

- - - - -
- - - -
- -
- - - - - - -
-
- - - - - - -
-
-
- -
-- - -
-- -
- -
- -
-
- -
- - -
- -

--
---
4.4.3 Listing poIicies using the IspoIicy command
Multiple named policies can be stored in the SONAS database. Policies can be listed with the
- command. Using - without arguments returns the name of all the policies
stored in the SONAS database. Specifying lists all the rules in a policy and
specifying - lists filesystems with applied policies. Example 4-14 shows the -
command output.
Example 4-14 Listing policies
- -
-

- -
-------
-----
--------
-----
---
- - -
-
- - - -

---
- -
- -
- -
---
4.4.4 Setting a poIicy as the active poIicy
A named policy stored in the SONAS database can be applied to a filesystem using the
- command. Policies set with the - command become the active policy for a
filesystem. The active policy controls the allocation and placement of new files in the
filesystem. The - command can also be used to remove an active policy for a
filesystem.
Example 4-15 setpolicy command output
- - - -
---
- -
- -
- - - -
---
4.4.5 Changing or repIacing an active poIicy
To replace an active policy's placement rules with the default GPFS placement rule, use the
setpolicy command with the option.
To change an existing active policy, you must use the setpolicy command with the option
as the first of three steps. This procedure is not recommended because it opens a window
where the desired policy configuration is not in place, and auto-migration or some other
unintended consequence might occur. The second step is to use the command to
modify the policy, and then use the - command to set the modified policy as the
active policy.
4.4.6 Running and stopping poIicies using runpoIicy command
The command executes or runs a policy on a filesystem. Either the default policy,
or the policy set on the filesystem using the - command, can be run by specifying
the option. Another policy stored in the SONAS database can be run by specifying the
option (see Example 4-16). The command executes migration and deletion rules.
Example 4-16 Sample runpolicy command output
- - -
- - -
- -
-

-
--
- -
- ---
-

- - - - -
- - - -
- -
--
- - - - - - -
-
-
- -
-- - -
-- -
- -
- -
-
- -
- - -
- -

-
--
- --
-
- - -
---
The - command stops running policy jobs depending on the parameters specified
(see Example 4-17). Auto-migrations can only be stopped by providing the time parameter.
Handle with care! Stopped auto-migrations get restarted within 2 minutes if the condition that
triggered this policy run (auto-migration policy applied against the file system - check with
- and then - <poIicy_name>) is still valid. Be also aware that stopping
auto-migrations can lead to out-of-space conditions that must be avoided under all
circumstances.
Example 4-17 Sample stoppolicy command output
- - -
---
4.4.7 Creating poIicies using mkpoIicytask command
The - command creates a SONAS cron job, a scheduled operation, which
applies the currently applied policy on a filesystem at a specified time. The -
command takes the filesystem as an argument. To remove scheduled policy tasks from a
filesystem you can use the - command with filesystem as the argument.
t is important that the task schedule allows sufficient time for the policy to complete before it
runs again, so that policy tasks do not overlap. For example, if the policy has rule specifying
that at 80% usage migrate to 75%, you are migrating 5% of the file system. Based on the
transfer rate between file system pools and the system load, you must schedule the policy
task with a sufficiently large time interval between start times to complete each migration
before the scheduled task submits another iteration of the migration policy.
Example 4-18 Sample mkpolicytask command
- - - -
- - ---
---
4.4.8 Peered poIicies
Peered policies contain placement rules only. Defines are generally not required for peered
LM policies. Placement rules select files by user-defined criterion or policy (Example 4-19).
Example 4-19 Sample placement rule
- --
- --
Peered pools must contain a default placement rule, that by default puts files in the lower
performance pool, and then select groups of files using rules for placement into the higher
performance pool. A sample placement rule is shown in Example 4-20.
Example 4-20 Sample default placement rule
- -
4.4.9 Tiered poIicies
Tiered policies contain both migration rules and optional placement rules. This type of policy
requires the defines contained in the sample TEMPLATE-LM policy. You can also
encapsulate weight expression as a define. Optional placement rules select files by policy.
Here we list best practices for migration rules:
Make sure that at least one threshold exists as a Safety Net, even if using other rules.
nclude exclusion clauses for migrated and special files in migration rules even if not using
HSM, so they can be added later.
Non-threshold migration needs an associated cron job to trigger it, as described later for
migration filters.
The policy is terminated by the default placement rule shown in Example 4-21.
Example 4-21 Default placement rule
- --
We used a default of a higher performance pool because subsequent tiering cascades data
from high performance to low performance pools.
4.4.10 HSM poIicies
Use the defines from the TEMPLATE-HSM rules. You can again encapsulate weight
expression as a define and optionally have placement rules to select files by policy.
Follow these best practices for the migration rules:
External pool rules: Use rules from template.
Threshold: Make sure at least one exists as a safety net even if using other rules.
Always include exclusion clauses (migrated, special files) in migration rules.
Non-threshold migration: This needs an associated cron job to trigger; you might want to
have a "time clause to prevent running on threshold trigger.
Levels: Define at least one rule for each migration "level (system pool2, pool2 hsm).
External pool rules: Use rules from template
Remember to terminate the policy with a default placement rule.
4.4.11 Remote caching poIicies
Following are considerations for remote caching policies:
Periodically runs parallel inodescan at home
Selects files/dirs based on policy criterion
ncludes any user defined metadata in xattrs or other file attributes
SQL like construct to select as shown in Example 4-22.
Example 4-22 Sample remote caching policy

Cache then pre-fetches selected objects
Runs asynchronously in the background
Parallel multi-node prefetch
Can callout when complete
4.4.12 PoIicy triggers
Policies can be applied to a filesystem or only reside in the SONAS database.
Filesystem policy:
- "active: One per filesystem, loaded from database (-)
Database policy
- "inactive: They are not running
- "default = Quick path to recalling a policy; it is a db state only
Triggers control when policies are activated. Policies only do something if triggered. We have
the following kinds of triggers:
Manual trigger:
- The runpolicy command allows a database policy to be run.
Automated triggers, also referred to as callbacks, triggered by a threshold:
- The SONAS GPFS file system manager detects that disk space is running below the
low threshold specified in the current policy rule, and raises a lowDiskSpace event.
- The lowDiskSpace event initiates a SONAS GPFS migration callback procedure.
- The SONAS GPFS migration callback executes the SONAS script defined for that
callback.
- The SONAS script executes the active filesystem policy.
Cron:
- n SONAS cron activates the default filesystem policy.
- Later releases might allow another database policy to be selected and not the default
policy for the filesystem.
When SONAS identifies that a threshold is reached, it triggers a new lowspace event every
two minutes so long as the fill level of the filesystem is above the threshold. SONAS knows
that a migration was already triggered, so it ignores the new trigger and it does not do any
additional processing, the migration that started earlier continues execution.
4.4.13 Weight expressions
Weight expressions are used with threshold migration rules. The threshold limits the amount
of data moved and the weight expression determines the order of files being migrated so that
files with the highest weight are moved first and until the threshold is satisfied.
Code the weight expression as a define because it makes rule easier to read, as the following
rule shows:
--- --
-- - -
-
Where weight expression is:
-- --
-- - --
--
The previous two statements are simpler to read than the combined statements:
--- --
-- -- -
-- --
- - -
4.4.14 Migration fiIters
Migration filters are used to control what gets migrated and when. Exclusion rules, or filters,
need to include the following files:
Migrated and special files: These must be used from the templates.
Optionally, small files: Leave small files behind for efficiency if they can fit on disk
(threshold + weight rule might do this anyway, so this might not be a useful rule).
The fine print: t means that small files are not migrated to offline storage, and cannot be
recovered from the offline storage. Although HSM can be used to recover files, it is not
desirable and is not supported as a customer action. Customers need to be using
backup/restore; in that case, if they run coupled with backup, the small files are backed up,
just not migrated.
Time filters can be useful when coupled with cron jobs, for example, running a cron every
Sunday at 4:05 AM; perhaps we are flushing a lot of files not accessed for a week.
4.4.15 GeneraI considerations
Understand your SONAS and Tivoli Storage Manager throughputs and loads, make sure your
thresholds leave sufficient freespace to finish without running out of disk space. Note that
bandwidth to Tivoli Storage Manager might only reduce the rate the filesystem fills during
peak usage and not necessarily at a fast enough rate depending on your configuration.
The filesystem high threshold must allow the peak use period to finish without filling the
filesystem 100%. Always use a threshold if you are using nformation Lifecycle
Management/HSM. Even if you do not expect to hit the threshold, this provides a safety net in
case your other policies have bugs, or in case your usage profile changes. Be aware that a
cron job that exploits a "low threshold rule causes "metadata spin. Migrations rules with no
threshold do not trigger automatically but need a cron job for that.
Tivoli Storage Manager clones backups if HSM migration is done first, migration still takes the
same amount of time to move data from SONAS to Tivoli Storage Manager, but backups
might be faster depending on server throughput. The - option can be set at
the Tivoli Storage Manager server and the option can be used to prevent the following
scenario:
f ACL data of a premigrated file are modified, these changes are not written to the Tivoli
Storage Manager server, if the file is to be migrated after this change. To avoid losing the
modified ACL data, use the option - -. This setting does not allow you to
migrate files whose ACL data was modified and for which no current backup version exists on
the server.
When using -, you must back up files or you might run out of space,
because HSM does not move files.
4.5 PoIicy creation and execution waIkthrough
We now illustrate the operational steps required to set up and execute SONAS policies, both
with the SONAS GU and with the SONAS CL.
4.5.1 Using the GUI versus using the CLI
n the following section, we explain how to use the GU. (Use of the CL is not covered here.)
4.5.2 Creating and appIying poIicies using the GUI
We first show how to create and apply a policy for an existing file system:
To create and apply a policy using the SONAS GU select FiIes FiIe Systems.
Select the file system and choose Actions Edit FiIe Systems as shown in Figure 4-9.
Figure 4-9 Edit the existing file system
The pop-up window shows an accordion menu. Select the PoIicy Text from the menu.
Click in the editor window and write the rules as shown in our example. Figure 4-10 shows
the Edit File System... window where the rules are entered.
Figure 4-10 Policy Editor window
Click to OK to save the changes and apply the policy.
RULE 'mpg' DELETE WHERE lower(NAME) LKE '%.mpg' AND FLE_SZE>20123456
RULE 'default' set POOL 'system'RULE 'default' SET POOL 'system'
The task progress window is displayed. Click CIose when the task is completed as shown
in Figure 4-11.
Figure 4-11 Apply policy task progress window
4.6 Information LifecycIe Management
n this section, we describe the nformation Lifecycle Management (LM) and provide
examples.
4.6.1 ExpIaining the ILM concept
SONAS implemented several tools that allow the administrator to increase the LM
efficiencies through powerful policy-driven automated tiered storage management. Using
these tools, GPFS can automatically determine where to physically store your data
regardless of its placement in the logical directory structure. File, file sets, file system pools,
and user-defined policies provide the ability to match the cost of your storage resources to the
value of your data. Using GPFS you can manage the following information:
Pools of storage
Files and file sets (you can also automate the management of file data)
Using these tools, GPFS can automatically determine where to physically store your data
regardless of its placement in the logical directory structure. Storage pools, filesets, and
user-defined policies provide the ability to match the cost of your storage resources to the
value of your data.
4.6.2 ILM tooIs
LM tools allow you to do the following functions:
Create storage pools to provide a way to partition a file system's storage into collections of
disks with similar properties that are managed together as a group. GPFS has three types
of storage pools:
- A required system pool that you create and manage through GPFS
- Optional user storage pools that you create and manage through GPFS
- Optional external storage pools that you define with GPFS policy rules and manage
through an external application such as Tivoli Storage Manager
Create file sets to provide a way to partition the file system namespace to allow
administrative operations at a finer granularity than for the entire file system.
Create policy rules based on data attributes to determine initial file data placement and
manage file data placement throughout the life of the file.
Storage pooIs
Physically, a storage pool is a collection of disks. Using storage pools, you can create tiers of
storage by grouping storage devices based on performance, locality, or reliability
characteristics. For example, one pool can be an enterprise class storage system that hosts
high-performance SAS disks, and another pool might consist of numerous disk controllers
that host a large set of economical SATA disks.
n our example, we defined several storage pools:
System pool, for metadata information
Silver pool, for files of large size
External pool, for near online files that are seldom accessed
There are two types of pools in SONAS, internal storage pools and external storage pools.
nternal storage pools are managed within SONAS. External storage pools are managed by
an external application such as Tivoli Storage Manager. For external storage pools, GPFS
provides tools that allow you to define an interface that your external storage manager uses to
access your data. GPFS does not manage the data placed in external storage pools. nstead,
GPFS manages the movement of data to and from external storage pools. Pools allow you to
perform complex operations such as moving, mirroring, or deleting files across multiple
storage devices, providing storage virtualization, and a single management context.
nternal GPFS storage pools are meant for managing online storage resources. External
storage pools are intended for use as near-line storage and for archival and backup
operations. However, both types of pools provide you with a method to partition file system
storage for considerations such as:
mproved price-performance by matching the cost of storage to the value of the data
mproved performance by:
- Reducing the contention for premium storage
- Reducing the impact of slower devices
- Allowing you to retrieve archived data when needed
mproved reliability by providing for:
- Replication based on need
- Better failure containment
- Creation of new pools as needed
InternaI storage pooIs
The internal GPFS storage pool to which a disk belongs is specified as an attribute of the disk
in the GPFS cluster.
GPFS assigns file data to internal storage pools under these circumstances:
When they are initially created; the storage pool is determined by the file placement policy
that is in effect when the file is created.
When the attributes of the file, such as file size or access time, match the rules of a policy
that directs GPFS to migrate the data to a different pool.
System pool
The system pool contains file system control structures, reserved files, directories, symbolic
links, special devices, as well as the metadata associated with regular files, including indirect
blocks, extended attributes, and so forth. The system pool can also contain user data. There
is only one system pool per file system, and it is automatically created when the file system is
created.
The amount of metadata grows as you add files to the system. Therefore, it is recommended
that you monitor the system pool to ensure that there is always enough space to
accommodate growth.
The system pool typically requires a small percentage of the total storage capacity that GPFS
manages. However, the percentage required by the system pool varies depending on your
environment. f the available space in the system pool begins to run low, you can increase the
available space by purging files or adding disks to the system pool.
User storage pool
All user data for a file is stored in the assigned storage pool as determined by your file
placement rules.
n addition, file data can be migrated to a different storage pool according to your file
management policies.
GPFS stores the data that describes the files, called file metadata, separately from the actual
file data in the system pool. You can create one or more user storage pools, and then create
policy rules to indicate where the data blocks for a file must be stored.
Managing storage pooIs
Managing your pools includes:
Creating pools
Deleting pools
Listing pools for a file system
Rebalancing files in a pool
Important: t is recommended that you use highly-reliable disks and replication for the
system pool because it contains system metadata.
Tip: To use a user-defined storage pool, a placement policy must be defined. By default, all
files are written in the system storage pool, and when it is filled, an out-of-space error
occurs even if the user-defined storage pools are empty.
Creating storage pools
The storage pool that a disk belongs to is an attribute of each disk and is specified as a field
in each disk descriptor when the file system is created using the - command or when
disks are added to an existing file system with the - command or changing the disk.
Storage pool names:
Must be unique within a file system, but not across file systems.
Cannot be larger than 255 alphanumeric characters.
Are case-sensitive. MYpool and myPool are distinct storage pools.
f a storage pool is not specified in the disk descriptor, the disk is by default assigned to the
system pool.
Creating a storage pool using the GUI
Follow this procedure:
1. Log on to your SONAS cluster and navigate to FiIe FiIe Systems menu.
2. Select your file system and from the Action menu select Edit FiIe System, or click with the
right mouse button to the file system and select the Edit FiIe System from the list as
shown in Figure 4-12.
Figure 4-12 Edit the file system
3. The list shows the default system storage pool in the list as shown in Figure 4-13.
4. Click the green plus sign to add a new pool.
5. Enter a name for the pool.
6. Select the disks from the drop-down list.
7. Allocate the capacity for the pool.
Figure 4-13 Creating a new pool
8. Click OK to save the changes.
9. Click CIose on the Status window when the progress bar and the Details window show
that the task has completed (see Figure 4-14).
Figure 4-14 Storage Pool creation status window
10.Next, click the plus sign to list the disk assigned to the storage pools as shown in
Figure 4-15.
Figure 4-15 Storage pool detailed view
Creating a storage pool using the CLI
n this section, we take you through creating a storage pool using the CL:
1. Execute the - command to list the storage pool in the system as shown in
Figure 4-23.
Example 4-23 Storage pool lists
-- -
-- - - - - -
- --
------
---
2. Execute the -- command to list the available disks in the SONAS cluster. Choose the
number of disks you want for the new storage pool, as shown in Example 4-24.
Example 4-24 List all disk in the SONAS cluster
-- --
--
- -
-- - --

-- - --

-- - --

-- --

-- --

-- --

-- --

-- --

-- --

-- --

-- --

-- --

---
3. Execute the - command to add the disk to a new pool (see Example 4-25). Use the
option to add the disks to a new pool.
Example 4-25 chdisk command example
-- -
------
-
-- --
- - -
-- --
- - -
-- --
- - -
- -- ---
---
4. Verify the changes, and execute the -- command to list the disks as shown in
Example 4-26.
Example 4-26 Verify disk changes
-- --
--
- -
-- - --

-- - --

-- - --

-- -

-- -

-- -

-- --

-- --

-- --

-- --

-- --

-- --

---
5. Add the disks to the filesystem using the - command with option as shown in
Example 4-27.
Example 4-27 Add disks to the file system
-- - -
------
-- - -
-- -

- -
- - -
-- - - -
- -
-- -- -
-- - -
-- -

- -
-- -- -
-- - -
-- -

- -
-- -- -
-
---
Verify the changes using the - command as shown in Example 4-28.
Example 4-28 verify pool changes
-- -
-- - - - -
-
- -
------
- --
------
---
Execute the - command to refresh the usage information (Example 4-29).
Example 4-29 lspool -u command example
-- -
-
-- - - - - -
- -
------
- --
------
---
DeIeting storage pooIs
n this section, we describe the circumstances under which you can delete storage pools.
System storage pool and user storage pools
Deleting the system storage pool is not allowed. n order to delete the system storage pool,
you must delete the file system.
n order to delete a user storage pool, you must delete all its disks.
Deleting a storage pool using the GUI
Proceed as follows:
1. Navigate to FiIes FiIe Systems, select the files system, and from the Action menu,
choose the Edit FiIe System as shown in Figure 4-16.
2. Click the slider and deallocate all space from the storage pool.
Figure 4-16 Delete storage pool
3. Click OK to save the changes.
4. Click CIose when the task is finished (see Figure 4-17).
Figure 4-17 Progress Window
5. The Storage Pool is now deleted. Review it in the File System List View as shown in
Figure 4-18.
Figure 4-18 File system list view
RebaIancing fiIes in a storage pooI
An administrator user can rebalance file data across all disks in a file system by issuing the
-- command, which has the following parameters:
Rebalances all files across all disks that are not suspended, even if they are
stopped. Use this option to rebalance the file system after adding, changing, or deleting
disks in a file system.
Migrates all data off suspended disks. t also restores all replicated files in
the file system to their designated degree of replication when a previous disk failure or
removal of a disk has made some replica data inaccessible. Use this option either
immediately after a disk failure to protect replicated data against a subsequent failure, or
before taking a disk offline for maintenance to protect replicated data against failure of
another disk during the maintenance process.
Migrates all critical data off of any suspended disk in this file system.
Critical data is all data that might be lost if currently suspended disks were removed.
Repairs the file placement within the file system pool. Files assigned
to one file system pool, but with data in a different file system pool, have their data
migrated to the correct file system pool.
Changes the replication settings of each file, directory, and system
metadata object so that they match the default file system settings. Next, it replicates or
unreplicates the object as needed to match the new settings. This option can be used to
replicate all of the existing files that had not been previously replicated or to unreplicate
the files if replication is no longer needed or wanted.
ExternaI storage pooIs
When you initially create a file, GPFS assigns that file to an internal storage pool. nternal
storage pools support various types of online storage. To move data from online storage to
offline or near-line storage, you can use external storage pools. External storage pools use a
flexible interface driven by GPFS policy rules that simplify data migration to and from other
types of storage such as tape storage.
You can define multiple external storage pools at any time using GPFS policy rules. To move
data to an external storage pool, the GPFS policy engine evaluates the rules that determine
which files qualify for transfer to the external pool. From that information, GPFS provides a list
of candidate files and executes the script specified in the rule that defines the external pool.
That executable script is the interface to the nformation Lifecycle Management for GPFS
external application, such as Tivoli Storage Manager (TSM), that does the actual migration of
data into an external pool. Using the external pool interface, GPFS gives you the ability to
manage information by allowing you to do the following tasks:
Move files and their extended attributes onto low-cost near-line or offline storage when
demand for the files diminishes.
Recall the files, with all of their previous access information, onto online storage whenever
the files are needed.
ExternaI pooI requirements
With external pools, GPFS provides metadata processing and the flexibility of using extended
file attributes. The external storage manager is responsible for moving files from GPFS and
returning them upon the request of an application accessing the file system. Therefore, when
you are using external storage pools, you must use an external file management application
such as Tivoli Storage Manager (TSM). The external application is responsible for
maintaining the file after it leaves the GPFS file system. For example, GPFS policy rules
create a list of files that are eligible for migration. GPFS hands that list to Tivoli Storage
Manager (TSM), which migrates the files to tape and creates a reference file in the file system
that has pointers to the tape image.
Tip: Depending on the configuration of your file system, if you try to enable file replication
in a storage pool having only one failure group, GPFS either gives you a warning or an
error message.
When a file is requested, it is automatically retrieved from the external storage pool and
placed back in an internal storage pool. As an alternative, you can use a GPFS policy rule to
retrieve the data in advance of a user request.
The number of external storage pools is only limited by the capabilities of your external
application.
GPFS allows you to define external storage pools at any time by writing a policy that defines
the pool and makes that location known to GPFS. External storage pools are defined by
policy rules and initiated by either storage thresholds or use of the command.
FiIe sets
A file set is a subtree of a file system namespace that in many respects behaves like an
independent file system. File sets provide a means of partitioning the file system to allow
administrative operations at a finer granularity than the entire file system.
Figure 4-19 shows an example of several file sets.
Figure 4-19 File set subtree example
The Fileset_u1 starts from /dir4/dir_b directory and has /user1/ as the parent directory.
The Fileset_u2, which starts at /user2 directory and has as its children the data1, data2,
data3, and data4 directories.
Additional information about file sets
The following general information applies to file sets:
File sets can be used to define quotas on both data blocks and inodes.
The owning file set is an attribute of each file and can be specified in a policy to control the
following items:
- nitial data placement
- Migration
- Replication of the file's data
File sets: Implementation
The following information applies to the implementation of file sets:
When the file system is first created, only one file set exists, which is called the root file
set. The root file set contains the root directory and any system files, such as quota files.
As new files and directories are created, they automatically become part of the parent
directory's file set.
The file set to which a file belongs is largely transparent for ordinary file access.
File sets: Namespace
The following information applies to the file set namespace:
A newly created file set consists of an empty directory for the root of the file set, and it is
initially not linked into the file system's namespace.
A newly created file set is not visible to the user until it is attached to the namespace by
issuing the - command.
File sets are attached to the namespace with a special link called a junction.
Only one junction is allowed per file set, so that a file set has a unique position in the
namespace and a unique path to any of its directories.
The target of the junction is referred to as the child file set, and a file set can have any
number of children.
After a file set is created and linked into the namespace, an administrator can unlink the
file set from the namespace by issuing the - command.
Unlinking makes all files and directories within the file set inaccessible. f other file sets are
linked below it, the other file sets become inaccessible, but they do remain linked and
become accessible again when the file set is relinked.
Unlinking a file set, like unmounting a file system, fails if there are open files. The
- command has a force option to close the files and force the unlink.
After a file set is unlinked, it can be re-linked into the namespace at its original location or
any other location.
The root directory of a GPFS file system is also the root of the root file set. The root file set
is an exception. The root file set is attached to the local namespace by using the standard
mount command. t cannot be created, linked, unlinked, or deleted by using the GPFS file
set commands.
File sets and storage pools
The following information applies to file sets and storage pools:
File sets are not specifically related to storage pools, although each file in a file set
physically resides in blocks in a storage pool.
The relationship is many-to-many:
- Each file in the file set can be stored in a separate user storage pool
- A storage pool can contain files from many file sets, however, all of the data for a
particular file is wholly contained within one storage pool.
Using file-placement policies, you can specify that all files created in a particular file set
are to be stored in a specific storage pool.
Using file-management policies, you can define how files in a specific file set are to be
moved or deleted during the file's life cycle.
File sets and snapshots
The following information applies to file sets and snapshots:
A GPFS snapshot preserves the content of the entire file system, including all its file sets,
even unlinked ones.
The saved file system can be accessed through the .snapshots directories; the
namespace, including all linked file sets, appears as it did when the snapshot was created.
Unlinked file sets are inaccessible in the snapshot, as they were in the active file system.
However, restoring a snapshot also restores the unlinked file sets, which can then be
relinked and accessed.
A file set is included in a snapshot if the snapshot is created after the file set was created.
Deleted file sets appear in the output of the -- command.
File sets and backups
The following information applies to file sets and backups:
Tivoli Storage Manager is unaware of the existence of file sets.
When restoring a file system that had been backed up to Tivoli Storage Manager, the files
are restored to their original path names, regardless of the file sets of which they were
originally a part.
During a full restore from backup, all file set information is lost and all files are restored into
the root file set.
Saving the output of the -- command aids in reconstruction of file set names and
junction locations.
A partial restore can lead to confusion if file sets were deleted, unlinked, or their junctions
moved, since the backup was made.
For more information, see Chapter 6, "SONAS administration on page 359.
4.7 WAN caching
This section provides an introduction to WAN caching and its management.
4.7.1 Managing Wide Area Network (WAN) caching
Wide Area Network (WAN) caching is a scalable, high performance remote file data access
function that is integrated with General Parallel File System (GPFS).
You can use WAN caching to distribute data transparently among Data Centers and multiple
remote locations without disruption to applications. WAN caching remote operations are
performed on the local system by one or more nterface nodes that are configured as gateway
nodes using SONAS CL commands, which use internal Network File System (NFS)
protocols to read and write data from nterface nodes in the remote home system.
Figure 4-20 provides an overview of a SONAS Wide Area Network (WAN) caching
configuration.
Figure 4-20 SONAS with Wide Area Network (WAN) Caching
The WAN caching functionality is enabled at the caching site by defining a file set and a
caching relationship from that file set to a file set that is exported over NFS at the home site,
specifying the P address of the NFS server and the share or export path.
All of the gateway nodes use the NFS protocol to mount paths that are exported from the
home site. A request on the cache file set is satisfied from cache if the data is already cached
at the local site. However, validation with the home file set is required to ensure that the
cached file contents are up to date with the home file set. This normally involves an NFS
request to get the file attributes, which contain the time when the file was modified at the
home system, and comparing them with attributes stored from previous verification. For
performance reasons, revalidation is not performed on every file access; revalidation is
performed only when an access request is initiated after a configured time interval passes.
This time interval is called the refresh lookup interval. The time delay between a write at the
cache file set and its corresponding update at the home file set is called the async delay;
maximum async delay latencies can be configured. The time interval between a read at the
cache file set and the last write at the home file set is called the validity lag; this attribute can
be set system-wide or per file set.
f the data is not already in cache, the request is sent to the gateway node. The gateway node
then retrieves the requested data over the WAN and stores it in the local GPFS file set, from
which the local application nodes subsequently access the data directly. Read requests are
split into multiple reads; the requested byte range is read synchronously while the remainder
of the file is read asynchronously. As soon as the synchronous read completes, that data is
served to the requesting NAS client while the remainder of the file is still being brought into
the cache asynchronously. Subsequent sequential reads of that file avoid the performance
overhead of a cache miss because these mounts remain in cache until the data is evicted. f a
NAS client synchronous read times out, the NAS client must attempt to perform the read
again. This might occur with a high latency WAN connection or during GPFS cache recovery,
where the file set is suspended and all of the requests are blocked until the recovery is
complete. The amount of time required for GPFS cache recovery is directly proportional to the
number of files in the file set.
WAN caching offers disconnected mode operations. f gateway nodes are disconnected from
the home file set, data that is already in cache remains in cache and continues to be served
to application nodes. f the requested data is not in cache, a file offline error is returned. WAN
caching can be configured to cause cached data to expire after the gateway nodes were in a
disconnected state for specified amount of time. This prevents access to stale data, where
staleness is defined by the amount of time that the WAN cache is out of synchronization with
data at the home site.
Cache file contents can be configured to be evicted when cache usage exceeds a soft or hard
quota limit on the caching file set. Cache eviction can be also triggered manually. There might
be a time lag between the time that an eviction is triggered and the time that the data is
actually evicted. Tuning the soft and hard quota limits minimizes application failure caused by
data being cached at a faster rate than it is being evicted.
WAN data transfer can be configured with or without a dedicated WAN link between the home
and cache file sets.
Requirements
A cache file set must be caching-enabled when it is created; existing file sets such as GPFS
version 3.3 file sets or file sets that were created before this feature is enabled cannot be
designated for WAN caching. File sets that were created using the - SONAS CL
command cannot be designated for WAN caching.
CapabiIities
WAN caching provides the following capabilities:
Configure and enable the SONAS WAN caching feature with or without a dedicated WAN
link.
Configurable caching behavior and modes.
Create, delete, and change query caching file sets in the SONAS environment.
Cached data can be accessed using NFS, CFS, HTTP and FTP protocols from SONAS
clients.
Seamless access to cached data even with SONAS node failover, disconnection or other
failures.
Uniform access to data from either the home or cache site using the same user D and
access control lists (ACLs).
Read data from, and write data to the cache.
Optimize utilization of WAN bandwidth and performance between the cache and home site
for reads and writes.
Cache sites can be smaller capacity than home sites.
Virtual machines can be served from SONAS cache.
Support multi-site scenarios:
- 1:N, Single home with multiple cache sites, 1 primary Read/Write (RW) cache site,
other sites Read Only (RO) for each file set.
- N:1, Multiple home sites, single cache site.
Antivirus applications on the cached data.
Support NDMP backup, backup over local area network (LAN), Tivoli Storage Manager
and Tivoli Storage Manager for Space Management in conjunction with caching functions.
Pre-populate the cache using policies to specify file selection.
Pre-populate the cache when the caching site is empty by initiating the process manually.
Provide consistent data sets at the home and cache sites by using snapshots.
Provide statistics including operations pending, operations completed, and health
information. Cache failures and changes to cache configuration automatically update the
SONAS health center.
Automatic expiration of cached data based on a configured expiration time latency, and
reporting of the expiration state.
Manage smaller capacities at cache sites by automatically evicting files based on a least
recently used (LRU) eviction algorithm.
4.7.2 WAN caching modes and data consistency
Data caching manages data consistency among versions of the same data across multiple
SONAS systems. Cross-system locking is not supported; potential data conflict is managed
by designating a caching mode at the file set level.
SingIe Writer mode
A file set in Single Writer mode has exclusive write permission to the data. No other cache file
set or NAS clients connected to any other system can be permitted to write to the data. f
there is more than one cache file set configured for access to the same file set, only one of
the cache file sets can be configured as Single Writer mode and the rest of the systems that
are caching that same data must be configured for Read Only mode (see next section) for
that file set. All of the cache file sets can read and cache the data, but only one cache file set,
the one that is configured in Single Writer mode, can write to it. This configuration prevents
potential data conflicts among systems.
Read OnIy mode
n Read Only mode, when data is accessed the data is fetched into cache as required, and it
can be read only by applications. t cannot be changed or deleted by applications because
data writes are not permitted.
LocaI Update mode
n Local Update mode, reads at the cache file set bring the data into the cache file set if it is
not already in cache. Writes on the cache file set, including the creation of new files, are not
sent to the home file set. Files that are updated at the cache file set become local to the
cache file set, and any further reads at the home file set are satisfied locally. n Local Update
mode, the files that are modified are not pushed back to the home file set.
4.7.3 WAN caching use cases
This topic provides an overview of some of the possible uses of WAN caching, including:
Multiple Remote Caches connected to a Central Data Repository, Multiple Remote Cache
Sites with Single Writer privilege, Write Back Cache to multiple Data Repositories/Archives,
and Cascading Content Distribution.
MuItipIe Remote Caches connected to a CentraI Data Repository
n this use case, a central site creates and maintains the data, including any updates (see
Figure 4-21). Only one primary Read/Write (RW) cache is supported, while other remote sites
read the data from the central site as Read Only (RO) if the data has not already been pulled
into the remote site's cache during a previous request.
Tip: Once a file set is configured in Local Update mode, its mode cannot be changed to
any other mode. A file set in Single Writer mode can be changed to Read Only mode, and
a file set in Read Only mode can be changed to Single Writer mode.
Figure 4-21 Multiple remote caches connected to a central data repository
Data can either be pulled on demand if it is not already resident in the local cache site, or data
can be prefetched at preset intervals. These intervals are based on policies that are
configured for each site by an administrator. You can optionally configure the pre-population
of caches by configuring policies to specify file selection. Pre-population of cache can also be
initiated manually.
A cache can be configured to discard data after a preset time that is configured by the
administrator. Expired data is sometimes referred to as stale or dirty data.
There is no mandatory minimum or maximum distance required between sites. A caching site
can be located in the same data center for performance reasons, or it can be located at an
extremely distant location. A request on an application node is served from cache if the data
is already cached locally. f the data is not in cache, the request is forwarded to the gateway
node. The gateway node then pulls the data over the WAN and stores it in the local cached
file set, where the application node can subsequently access the data directly.
MuItipIe remote cache sites with singIe writer priviIege
This use case allows each site to own one or more file sets, where only one site owns Single
Writer mode on page 187 to any particular file set. As data is written at the remote sites, it is
sent to one or more data centers for archiving or disaster recovery purposes or for
redistribution to other remote sites. Data is continually sent asynchronously to the central
repository using a background process. Data can then be sent from the data center repository
to other remote sites, which have Read Only (RO) access.
Write back cache to muItipIe data repositories/archives
n this use case, data at remote sites is the primary data and is asynchronously written to one
or more data centers for backup and/or disaster recovery. Caching systems have the most
up-to-date data and the home file sets are used primarily as a backup and archive site.
Data from remote sites can be split at the file set level and sent to two or more data centers to
reduce bandwidth when serving clients, for example in a disaster recovery mode (see
Figure 4-22).
Figure 4-22 Write back cache to multiple data repositories/archives
n the event of a failure of a home file set, the cache is retargeted to another home file set,
and data must be resynchronized between the cache and the new home file sets using the
CL command. Similarly in the case of a caching site failure, a new cache site can
point to the home site that contains the most recently backed up data of the failed cache site.
Remote caching sites can be used for storing VM virtual disks, and can provide consistent
asynchronous WAN replication between sites. Snapshots created at the cache site send the
data from the snapshot to the home site, where a peer snapshot is also created after all of the
data is received, resulting in identical snapshots at both locations. Cache sites can perform
functions such as backup and antivirus protection that are also available at single home sites.
Cascading content distribution
This use case allows for the distribution of content, including video and images, from a central
site to a remote site, and from one remote site to another remote site. Data can also be
spread across data centers before being cascaded, but only one site has write ownership.
Branch office cache
n this use case, a company has more than one branch office or site. Each site has a
dedicated storage, but the branch sites do not want to manage the storage or backups. When
the network connection between the central and the remote site is lost, the branch wants to
continue to access data in local storage for some predetermined amount of time, and limit
access after a period of acceptable staleness elapses.
Figure 4-23 Branch office cache
The following two cases differ in the capacity of the cached site relative to the home site:
1. The cache site is smaller than home site. All of the data is housed in a large central site; a
predefined amount of that data must be accessed at the branch site. As data is updated in
the local branch office system, the changes are updated asynchronously at the central
site. At the central site, a central backup mechanism is used to maintain backup while
users continue to update their home folders.
2. The cache site and home site each have SONAS systems of similar sizes. Both sites
cache their data from the site that owns that data to the other site for faster access to the
data. Data is cached either at the time of access or pre-populated before access.
4.7.4 Managing caching gateway nodes
To enable Wide Area Network (WAN) caching on a SONAS system, use the
SONAS CL command to configure specified nterface nodes in that system to function as the
caching gateway nodes that exchange data with other systems. You can verify that the
gateways are configured correctly by using the lsnode SONAS CL command, which indicates
which nodes are caching gateway nodes. The SONAS CL command removes
the caching gateway function from specified nodes.
To specify a system when using any of these three commands, use the -c or --cluster option
of the command and specify either the system D or the system name. f the -c and --cluster
options are omitted, the default system, as defined by the -- command, is used.
EnabIing the caching gateway function on nodes
To enable the WAN caching function on nterface nodes, submit the SONAS
CL command using the --nodelist option to specify either the node names or node P
addresses in a comma-separated list, or to specify a network group, as in Example 4-30.
Example 4-30 Enable the caching gateway function on nodes
-- - --
---
Listing nodes
To determine which nodes were configured as caching gateway nodes, use the - CL
command with the option, as in Example 4-31.
Example 4-31 Listing nodes
-
-------
- -- -- ---- -
-- -- -
-- -
---
----

---
-
--
-----
-
--
-----
-

-----
-

The nodes with yes in the cache column are caching gateway nodes.
Tip: f a network group is used to configure the caching gateway nodes, the nodes that are
members of that network group at the time that the command is submitted are configured
as caching gateway nodes. f that network group membership changes after the nodes
were configured as caching gateway nodes, the network group membership change does
not affect which nodes are configured as caching gateway nodes, which remain configured
as originally defined by the SONAS CL command.
Tip: The option specifies display of the command output as colon-delimited fields. You
might want to use the -r option to force a refresh of the nodes data before it is displayed;
otherwise, the displayed information might be stale. Output similar to the following is
displayed:
Removing the caching gateway function from nodes
To remove the caching gateway function from nodes, specify either the node names or node
P addresses in a comma-separated list or specify a network group, using the
CL command, as in Example 4-32.
Example 4-32 Removing the caching gateway function from nodes
- -
- -
- -
---
You can use the -f or --force option to prevent the normal manual removal confirmation prompt
from being displayed.
4.7.5 Managing caching source shares and exports on the home fiIe set
You must first create a caching-enabled share or export on the home file set using the
- SONAS CL command before you can cache that share or export in a client
system. You can use the -- CL command on the home file set to list the
caching-enabled shares and exports on that system. You can use the -
command to remove a cache-enabled share or export, and you can use the -
command to change a cache-enabled share or export configuration on the home file set, for
example to add or remove a caching client system.
To specify a system when using any of these four commands, use the -c or --cluster option of
the command and specify either the system D or the system name. f the -c and --cluster
options are omitted, the default system, as defined by the setcluster command, is used.
Creating a caching source share or export
To enable WAN caching for a file set, specify a share or export name and an existing file
system path with the - SONAS CL command. Also specify a
comma-separated list of management P addresses with the --client option, one for each of
the systems that you want to enable to cache the file set, and follow each P with either (rw) or
(ro) to specify whether the system managed by that P address can access the file set in
read/write or read only mode. The entire parameter value must be enclosed by the single
quote character at each end. Only one system can be defined to access the file set in
read/write mode at any particular point in time. n Example 4-33, the share name is trial_ro
and the file system path is /ibm/gpfs0/trial_ro.
Example 4-33 Creating a caching source share or export
- -
---
Listing caching source shares and exports
To show information about caching sources that are configured on a home system use the
-- CL command, as in Example 4-34.
Example 4-34 Listing caching source shares and exports
--
- -
-- -
-
-
---
To show information about all of the home shares and exports that are configured for a
particular client system, use the --clientclusterid option and specify the system D of the client
system, as in the following example:
Lists all the wan-caching shares and exports for which the passed system id is a client.
Output similar to the following is displayed:
You can use the --details option to display whether the system is actively using the cache
share or export, as in Example 4-35.
Example 4-35 Listing detailed caching source shares and exports
-- -
-
-
-
----
------
-----
-
---
Only the listed P addresses are permitted to access the corresponding share or export.
You can use the -Y option to specify display of the command output as colon-delimited fields.
Removing a caching source share or export
To remove a cache source file set configuration on a home file set and delete the share or
export, use the - CL command and specify the share or export name to be
removed. You must respond yes to the confirmation prompt to remove a cache source file set
configuration, as shown in Example 4-36.
Example 4-36 Removing a caching source share or export
-
- -
---
Changing a caching source share or export
To add a caching client system to a configured caching share or export on the home file set,
specify the share or export name using the - CL command and use the
--addclient option to specify the management P address of the caching system that you are
adding, suffixed by (rw) or (ro) for its access mode, with the entire parameter value enclosed
by the single quote character at each end, as in Example 4-37.
Example 4-37 Changing a caching source share or export
- -
---
This command option must also be used after changing a cache system D to enable
configuration or reconfiguration of client caches using the new D.
To remove a caching client system from a configured caching share or export on the home
file set, specify the share or export name using the - CL command and use
the --removeclient option to specify the client system D to be removed. The client system D
is displayed by the lswcachesource CL command. f there is only one client system
configured for a share or export, it cannot be removed with the chwcachesource CL
command.
You can use the -u or --updateKeys option to update the WAN cache internal user keys using
the client WAN cache's user key definitions. One scenario where it is required is when the
system D of a client system is changed.
You can use the -f or --force option to force the command submission while preventing any
normal output display.
4.7.6 Managing cache fiIe sets
Before you can create a cache file set on a client system using the SONAS CL
command, you must first create the caching-enabled share or export on the home file set
using the - SONAS CL command, and enable Wide Area Network (WAN)
caching on the client SONAS system using the SONAS CL command to
configure specified nterface nodes in that system to function as the caching gateway nodes
that exchange data with other systems.
You can use the - CL command to list the cache file sets on a system. You can use
the command to remove a cache file set. You can use the command to
change the configured parameters of a cache file set. You can use the command to set
certain cache attributes at the client cache system level, and you can use the
command to perform caching maintenance operations on the cache file set. You can use the
-- command to display the status of gateway nodes and client shares and
exports, and to list pending and completed cache operations.
To specify a system when using the , -, and commands,
use the -c or --cluster option of the command and specify either the system D or the system
name. f the -c and --cluster options are omitted, the default system, as defined by the
setcluster command, is used.
Creating a cache fiIe set
To create a cache version of a file set on a client system, use the SONAS CL
command and specify a device, a file set name, and file system junction path. For device,
specify the device name of the file system that you want to contain the new cache file set on
the client system. The device name of the file system does not need to be fully qualified; for
example, fs0 is as acceptable as /dev/fs0. The device name must be unique within a GPFS
cluster, and the device cannot be an existing entry in /dev. The file system junction path
specifies the name of the junction, must be a file system path, and cannot refer to an existing
file system object. f the file system junction path is not specified, the default is file system
mount point/file system name.
Tip: f the home file system or file set is destroyed and recreated or restored at the home
site, the cache relationship is lost. t must be re-established by submitting the
CL command and specifying the path using the --remotepath and --homeip options.
You must also specify the home system P address and path of the file set cache source by
using the --remotepath required option of the SONAS command in the format home
system P address:exported path (see Example 4-38). t is the share or export that you want
to cache to the remote file set; this share or export must have previously been created on the
home file set using the - SONAS CL command. The home system P address
is a public P address of the home system. f the home system P address is not specified
(which requires that the colon separator is also omitted), an internal public P of the home
system is selected based on a round robin algorithm. The exported path can be either the full
path specification or just the name of the exported file set. n the first instance of the following
example, the home system P address is specified; in the second instance it is omitted.
Example 4-38 Creating a cache file set
-
-
You must also use the --homeip required option to specify the P address of the active
Management node of the home system.
You must specify either the -h option to specify the hard limit of disk usage for the file set, or
the -s option to specify the soft limit. These values are used for automated cache eviction.
You can use the --cachemode option to specify the WAN caching mode of the cache version
of the file set that you are creating on the remote system. Valid values are read-only,
local-updates and single-writer. f the option is not used or its value is not specified, the
default value is read-only.
Once the cache mode is set to local-updates, you cannot change the cache mode to any
other mode value.
To change the cache mode of a file set, the file system that contains the cache file set to be
changed must be unmounted.
See"WAN caching modes and data consistency on page 206 for more information.
n Example 4-39, the device is gpfs0, the file set name is test_2, and the file system junction
path is /ibm/gpfs0/test_2:
Example 4-39 -mkcache with homeip option
- - --
-
---
You can use the -i option to specify the maximum number of inodes that can be allocated and
the number of inodes that the system immediately preallocates to the cache version of the file
set, separated by a colon. You can use the multiplicative suffixes M for millions and either K or
k for thousands; for example, -i 1M:500K.
Tip: You cannot link any file set, whether dependent or independent, into a cache file set.
Note:f the file set is being created with single-writer caching mode, the system verifies
whether the home file set share or export allows the specified remote cache file set to be in
single-writer mode. f write permission has not been configured for the cache file set, an
error message is displayed.
You can use the --FileOpenRefreshnterval option to specify the number of seconds since the
last time that the file was opened, after which revalidation with the file version on the home file
set must be performed. Similarly, you can use the --FileLookupRefreshnterval option to
specify the number of seconds since the last time that a file lookup has occurred, after which
revalidation with the file version on the home file set must be performed. A file lookup is when
the file metadata is retrieved without actually opening the file for a read or write, which usually
occurs when displaying or reporting the file metadata. The --remotepath
10.0.0.41:/ibm/gpfs0/wcacheSource --remotepath /ibm/gpfs0/wcacheSource
--DirOpenRefreshnterval and --DirLookupRefreshnterval options specify the corresponding
time latencies for directories rather than files.
You can specify a time interval in seconds after which queued updates are flushed when a
gateway node becomes disconnected by using the --DisconnectTimeout option. A value of
disable for this option forces synchronous operations. You can specify a time interval after
which files and directories in the cache file set expire when the cache file set is in
disconnected mode by using the --ExpirationTimeout option.
This option is disabled by default. You must enable this option by specifying a value if you
want to use the CL command --expire option to mark all of the cache file set
contents as expired. You can specify the number of seconds before a queued write must be
sent to the home file set by using the --AsyncDelay option. This delay might be useful for write
intensive applications that frequently write to the same file set. Delaying writes to the home
file set provides the opportunity to send a single write containing the most current version of
data resulting from multiple writes, thus reducing WAN traffic. However, setting a higher value
decreases the currency of data on remote system.
When creating a cache file set, if any of the following options or their corresponding values
are not specified, the default value listed in Table 4-1 on page 215 is used:
Table 4-1 Cache file set options
You can specify a comment that is displayed in the output of the - CL command by
using the -t option. The comment must be less than 256 characters in length. f the comment
text contains a space character, the comment text must be enclosed in quotes.
Tip: Setting a lower value for a refresh interval provides greater consistency between the
home and remote system data. f the home file set data changes frequently, you might
want to set refresh intervals as low as 0 for critical data consistency.
Option DefauIt vaIue
CacheMode 1 (Read-only)
fileOpenRefreshnterval 30 seconds
fileLookupRefreshnterval 30 seconds
dirOpenRefreshnterval 60 seconds
dirLookupRefreshnterval 60 seconds
asyncDelay 15 seconds
DisconnectTimeout 60 seconds
expirationTimeout disabled
i (max inodes & # inodes to preallocate 100,000
Listing cache fiIe sets
To show information about cache file sets on a remote system, use the - CL
command and specify a device name, which does not need to be fully qualified, as in
Example 4-40.
Example 4-40 Sample lswcache command
- -
- -
- --
--
-
- -
- --
--
-
- -
You might want to use the -r option to force a refresh of the cache file sets data before it is
displayed; otherwise, the displayed information might be stale.
You can use the -v or --verbose option to also display this information for a cache file set:
Rootnode (The number of the root inodes)
ParentD (The parent file set identifier; -- displays if none exists.)
async delay
file open refresh interval
file lookup refresh interval
directory open refresh interval
directory lookup refresh interval
expiration timeout
disconnect timeout
Removing a cache fiIe set
To remove a cache file set from a remote system and delete its contents after unlinking any
child file sets, use the CL command and specify the file system name and the file
set to be removed. You must respond yes to the confirmation prompt to remove a cache file
set, as in Example 4-41.
Example 4-41 Sample rmwcache command output
-
-
- -
---
This command cannot be used to remove a file set that is not a cached file set.
Changing a cache fiIe set
To change a cache file set configuration on a remote system, specify the device and file set
name using the CL command. As with the CL command, the device
name of the file system does not need to be fully qualified; for example, fs0 is as acceptable
as /dev/fs0. The device name must be unique within a GPFS cluster.
You can change the following values of CL command options using the
CL command:
-t (to change the comment)
--cachemode
--FileOpenRefreshnterval
--FileLookupRefreshnterval
--DirOpenRefreshnterval
--DirLookupRefreshnterval
--ExpirationTimeout
--DisconnectTimeout
--AsyncDelay
--remotepath
--homeip (This option must be specified when the --remotepath option is specified.)
-i (maximum number of inodes and maximum number of inodes to preallocate)
Once the cache mode is set to local-updates, you cannot change the cache mode to any
other mode value.
To change the cache mode of a file set, the file system that contains the cache file set to be
changed must be unmounted.
Example 4-42 changes the home P of a single-writer mode file set named finalTest to a
different system.
Example 4-42 Sample chwcache command syntax
- - -
You must respond with yes to submit the change request. For more information, see Move
home system of a single-writer mode file set to a different system on page 227.
You can use the -f or --force option to force the command submission while preventing any
normal output display.
You can display the cache file set configuration by using the - CL command.
Tip: f the file set is being created with single-writer caching mode, the system verifies
whether the home file set share or export allows the specified remote cache file set to be in
single-writer mode. f write permission has not been configured for the cache file set, an
error message is displayed.
Setting seIect cache attributes at the system IeveI
You can use the CL command to set the following cache attributes at the remote
cache system level:
--cachemode
--FileOpenRefreshnterval
--FileLookupRefreshnterval
--DirOpenRefreshnterval
--DirLookupRefreshnterval
--ExpirationTimeout
--DisconnectTimeout
--AsyncDelay
Performing caching maintenance operations on the cache system
You can use the CL command specifying the device and cache file set name to
resynchronize client cache data to the home file set, to flush the cache /O to the home file
set, to evict cached data, and to expire or unexpire all of the data in a file set.
You can use the --resync option when there are inconsistencies between the home and cache
versions of data to ensure that the home version is consistent with the cache version.
You can use the --flushqueue option to flush a client file set's /O queue to the home file set.
The command completes when the queue is empty. You can monitor the queue by using the
--connectionstatus option of the lswcachestate CL command.
You can use the --expire option to mark all of the cache file set contents as expired.
You can use the --unexpire option to remove the expired status from all of the cache file set
contents that are marked as expired.
You can use the --evict option to remove cache data if the soft limit is set for the disk usage by
the file set, either when the cache file set is created using the CL command, or
subsequently using the - CL command with the -j option specifying the file set. You
can use the - CL command with the -j option specifying the file set to display the
current soft limit. You cannot use the --evict option if the soft limit is not defined, or is zero. f
you use the --evict option you must specify the --evictsafelimit suboption. The --evict option
takes the following suboptions:
Tip: f an attribute is not specified at the file set level and the same attribute is defined at
the system level using the chcfg CL command, the system level attribute value is used.
When both file set level and system level definitions are present for the same attribute, the
file set level attribute value has precedence over the system level attribute value.
Tip: The --expire and --unexpire options fail if the --ExpirationTimeout option of the cache
file set is disabled, which is the default value.
f you use the --evict option, you must use the --evictsafelimit suboption and specify the
target quota limit for eviction that is used as the low watermark. The value specified with
the --evictsafelimit suboption must be less than the value for the soft limit, which can be
displayed using the - CL command with the -j option specifying the file set. The
value specified by the --evictsafelimit suboption overrides the soft limit value during the
manual eviction operation.
You can use the --log suboption and specify the location of the eviction log file. By default,
the logged information is appended to mmfs.log
You can use the --evictorder suboption and specify either LRU or SZE to indicate the
eviction queue ordering sequence.
You can use the --evictfilenamepattern suboption and specify a value that is a file name
pattern regular expression.
You can use the --evictminfilesize suboption and specify the minimum file size in KB.
You can use the --evictmaxfilesize suboption and specify the maximum file size in KB.
DispIaying WAN caching status
You can use the -- CL command and specify a file system to display the
connection status of client cache shares and exports, and to list the status of pending and
completed cache operations. You can optionally specify a file set name to limit the display to
the specified file set.
You can use the --connectionstatus option to display the status for all of the cache gateway
nodes and client shares and exports. Displayed connection status includes the remote file set
status, the cache gateway assigned to the specified file set, the status of the gateway, and the
status of the queue.
Remote fiIe set status can have one of the foIIowing vaIues:
Active: The cache is active and the NFS mount from home was successful. The value
Active is valid for all caching modes.
nactive: The cache has not connected to home. This state changes when new operations
are initiated that require cache to contact home.
Unmounted: The cache cannot mount NFS from home.
Disconnected: The cache cannot connect to home.
Expired: The cache is disconnected from home after expiration timeout set for the cache.
NeedsResync: The cache has detected inconsistencies between home and cache, and
these inconsistencies are being fixed automatically. This state is transitional.
Dirty: The cache has data that is not sent to home.
Cache gateway status can have one of the foIIowing vaIues:
Active: The queue is active.
FlushOnly: The queue awaits application requests.
QueueOnly: Operations are queued but not flushed. t is a temporary state.
Dropped: A transient state is due to external changes.
Recovery: A WAN cache recovery is in progress.
Example 4-43 shows the format of the --connectionstatus option.
Example 4-43 List Cache gateway status
-- - --
- - - -- -

-- - -- -
- - -- -
You can use the --actionstatus option to display the cache operations currently pending and
the cache operations that have completed since the most recent startup. Possible status
values include RUNNNG, COMPLETED and FALED. A status of RUNNNG indicates that
the task is still progressing in the background. A status of COMPLETED indicates that the
action was performed successfully. A status of FALED indicates that the task did not
complete, and a possible error reason is reflected in the output.
Example 4-44 shows the format of the--actionstatus option.
Example 4-44 Sample lswcachestate command with actionstatus option
-- - --
- -- - -- -
-- -
- -- - - - - -
- -
- - - - - - -
- -
--- - -
-- - - -
--- - --
-- - -
- --- - --
You can use the -Y option to specify that the output is displayed in colon-delimited format.
4.7.7 Managing cache prepopuIation
Wide Area Network (WAN) caching fetches complete files on demand from the home file set
to the remote system cache in real time during normal operation. The prepopulation feature
moves files into the cache in batch mode so that they are already present in the cache when
they are accessed by an application. Prepopulation of cache before an application is started
can reduce the network delay when the application starts. Prepopulation can also be used to
proactively manage WAN traffic patterns by moving files over the WAN during a period of
lower WAN usage in anticipation that it might otherwise be accessed during a period of higher
WAN usage. You can use the r CL command specifying a file system, file set and
policy to prepopulate cache. You can use the - CL command specifying a file system
and file set to display the status of a prepopulation operation.
To specify a system when using the and - CL commands, use the -c or
--cluster option and specify either the system D or the system name. f the -c and --cluster
Running cache prepopuIation
To cache all of the data in a file set from the home file set to its cache file set on the remote
system, use the CL command, specifying the file system, the file set name, and
the policy to use for prepopulation. The specified policy can only use the LST rule and cannot
contain any REPLCATE, MGRATE or DELETE rules. See "Creating and managing policies
on page 179 for more information. Prepopulation performs revalidation of locally stored
cached file attributes at the remote system against the corresponding home file set attributes
and only fetches a file if the locally stored cached file attributes differ from those of the
corresponding file at the home file set.
DispIaying cache prepopuIation status
To display the status of a prepopulation operation, use the - CL command. You can
optionally limit the display to a file system using either the -d or --filesystem option and you
can optionally limit the display to a file set using either the -f or --filesetName option. You can
also use the --latestPrePops option to limit the display to the specified number of the most
recent prepopulation status entries, as in Example 4-45.
Example 4-45 Displaying cache prepopulation status
-
- -- - - -- - -
-
-
-
You can use the -Y option to specify display in colon-delimited format, as in Example 4-46:
Example 4-46 Displaying cache prepopulation status with -Y option
-
-----
-------
--

--

4.7.8 Managing cache fiIe set peer snapshots
The cache file set peer snapshot function provides a generic infrastructure for obtaining an
application consistent point in time copy of data in the cache file set that can be integrated
with other functions to meet specific customer requirements such as backup and recovery.
The peer snapshot pushes all of the snapshot data in cache to the home file set so that the
home data is consistent with cache, and then takes a snapshot of the corresponding home
data. This results in a pair of peer snapshots, one each at the cache and home file sets, that
refers to the same consistent copy. Peer snapshots can be created manually on demand or
on a defined schedule.
To specify a system when using the peer snapshot commands, use the -c or --cluster option
Note:f cache is disconnected from the home file set when the cache snapshot is created,
the cache notes that the peer snapshot on the home file set has not been created. When
the home file set becomes reconnected to cache, the cache attempts to resume the
creation of the missing peer snapshot at the home file set if conditions permit. Certain
events in the intervening time interval might prevent resuming the creation of the peer
snapshot at the home file set.
Creating a peer snapshot
To create a pair of peer snapshots, one each on the remote and home file set, use the
- SONAS CL command and specify a file system, a file set name, and peer snapshot
name. For file system, specify the device name of the file system that you want to contain the
peer snapshot on the cache file set. The device name of the file system does not need to be
fully qualified; for example, fs0 is as acceptable as /dev/fs0. The device name must be unique
within a GPFS cluster, and the device cannot be an existing entry in /dev. The file system
must be unique within a device. The peer snapshot file name is created by prefixing the
specified peer snapshot name with system-generated values using the format specified peer
snapshot name-psnap-system uid-file system uid-file set uid-timestamp.
The maximum number of snapshots per file set is 100. Example 4-47 creates a peer
snapshot named test on the file system named gpfs0 and file set named sw_fset.
Example 4-47 Creating a peer snapshot
- - -- -
-- - -
- --
- -
---
DispIaying peer snapshot information
To list peer snapshot information, use the -- SONAS CL command. You can optionally
specify a file system; if no file system is specified, information for all peer snapshots on the
system is displayed. f you specify a file system, you can f the maximum limit of psnaps is
reached, this automatically deletes the oldest snapshot.also optionally specify a file set; if you
specify a file system but no file set, information for all peer snapshots on the file system is
displayed.
You can use the -r or --refresh option to force a refresh of the peer snapshot information
before it is displayed; otherwise, the displayed information might be stale. When you use this
option, a message is displayed before the peer snapshot information is displayed, as in
Example 4-48.
Example 4-48 Displaying peer snapshot information
-
-- - - - -
- -- -
- --
---
A status of valid indicates that there is a corresponding peer snapshot on the home file set. A
status of invalid indicates that a corresponding peer snapshot does not exist on the home file
set.
You can use the -u or --usage option to include usage data in the display of the peer snapshot
information. This usage data is similar to the usage data that is displayed by the ---
CL command. A message is displayed before the data is displayed to indicate that the peer
snapshot information and usage data is refreshed, as in Example 4-49.
Example 4-49 List peer snapshot status
--
-
-- - - - - -
-
- -- - - --
---
You can use the -v option to include both the system D and usage data in the display of the
peer snapshot information. This usage data is similar to the usage data that is displayed by
the --- CL command, as in Example 4-50.
Example 4-50 List peer snapshot status
--
-- - - - - -
- - -- -
- --
---
Because neither the -r or --refresh options nor the -u or --usage options are specified in the
above example, the displayed information might be stale.
Removing a peer snapshotl
To remove a peer snapshot pair, use the - CL command and specify a file system, a
file set name, and peer snapshot name. For file system, specify the device name of the file
system that contains the peer snapshot that you want to remove on the cache file set. The
device name of the file system does not need to be fully qualified; for example, fs0 is as
acceptable as /dev/fs0. The device name must be unique within a GPFS cluster. The file
system must be unique within a device. You must respond yes to the confirmation prompt to
remove the peer snapshot pair, as in Example 4-51.
Example 4-51 Removing a peer snapshot
- - -- -
- -
- - - ---
---
ScheduIing automated cache fiIe set peer snapshot tasks
Peer snapshot tasks can be created using the -- CL command to automatically
submit peer snapshot creation requests on a defined schedule. These tasks can be changed
using the -- command or removed using the -- command, and
information about these tasks can be displayed using the --- CL command.
To specify a system when using the peer snapshot commands, use the -c or --cluster option
Tip: Because the -u or --usage option performs the same peer snapshot information
refresh as the -r or --refresh option, and additionally refreshes and displays usage
information, you cannot specify both options in the same command submission instance.
Creating a peer snapshot task
To create a "create peer snapshot task, use the -- SONAS CL command and
specify a file system, a file set name, and task name. For file system, specify the device name
of the file system that you want to contain the peer snapshot on the cache file set. The device
name of the file system does not need to be fully qualified; for example, fs0 is as acceptable
as /dev/fs0. The device name must be unique within a GPFS cluster, and the file system must
also be unique within a device.
Use either the --hournterval or the --minutenterval option, or both, to specify the time latency
since the last submission of the task for the next subsequent task submission. Use the
--hournterval option to specify the number of hours, which can be any non-negative integer. f
the value is zero or the option is not specified, the --minutenterval option must be specified
with a non-zero value. Use the --minutenterval option to add the specified number of minutes
to the value of the --hournterval option. Valid values are 0, 15, 30 and 45; however, if the
--hournterval option is not specified or is specified with a value of zero, the --minutenterval
option must be specified as one of the following non-zero integers: 15, 30 or 45.
You can use the --psnapLimit option to specify the number of peer snapshots to be retained
for a file set; the maximum is 100 and the default when the option or its value is not specified
is 10. When the limit is exceeded, the oldest snapshot of the file set is removed when a new
snapshot is created.
A peer snapshot task named task that is automatically submitted every four hours and 45
minutes is created with the command in Example 4-52 for the file set named sw_fset on file
system named gpfs0.
Example 4-52 Creating a peer snapshot task
-- - -- -
- ---
Changing a peer snapshot task
To change a create peer snapshot task, use the -- SONAS CL command and
specify the task name. You can change the --hournterval, --minutenterval and --psnapLimit
values of the task subject to the limitations described in the preceding section that describes
the usage of the -- SONAS CL command. Example 4-53 changes the peer
snapshot task named task to be submitted every three hours, and specifies that the system
retain up to a maximum of 25 versions of these peer snapshots.
Example 4-53 Changing a peer snapshot task
-- - -
---
Note:The initiation of a snapshot creation instance by a 'create peer snapshot' task occurs
only at 00, 15, 30 and 45 minutes after the hour. A snapshot for a scheduled interval might
not be created by a 'create peer snapshot' task if the specified time latency has not passed
since the completion of the most recent snapshot creation by that task.
You can use the --suspend option to prevent automatic submission of scheduled peer
snapshot tasks, as in Example 4-54.
Example 4-54 Changing a peer snapshot task using --suspend option
-- - --
---
The task status is displayed as Suspended by the --- SONAS CL command as long
as the task is in the suspended state.
You can use the --resume option to change the status of a suspended scheduled peer
snapshot task to Active so that automatic submission of its scheduled peer snapshot tasks
resumes, as in Example 4-55.
Example 4-55 Resume a peer snapshot task
-- - -
---
DispIaying peer snapshot task information
To display information about peer snapshot tasks, use the --- SONAS CL
command. You can optionally specify a file system; if no file system is specified, information
for all peer snapshot tasks on the system is displayed. f you specify a file system, you can
also optionally specify a file set; if you specify a file system but no file set, information for all
peer snapshot tasks on the file system is displayed. Example 4-56 displays all of the peer
snapshot task for the specified system.
Example 4-56 Displaying peer snapshot task information
---
- -- - - -
- - --
---
You can use the -v or --verbose option to include the system D and last date and time that the
task was submitted in the information that is displayed, as in Example 4-57.
Example 4-57 Displaying verbose peer snapshot task information
---
- -- - - - -
-
- - --
---
You can use the -Y option to specify display of the command output as colon-delimited fields,
as in Example 4-58.
Example 4-58 Displaying peer snapshot task information in semi colon-delimited output
---
------------
---------

Removing a peer snapshot task
To remove a peer snapshot task, use the -- CL command and specify the peer
snapshot task name. You must respond yes to the confirmation prompt to remove the
specified peer snapshot task, as in Example 4-59 that removes the peer snapshot task
named task:
Example 4-59 Removing a peer snapshot task
-- -
- -
---
Moving singIe writer mode of a fiIe set to a different cIient system
The following example scenario describes how to dynamically change the designated single
writer role of a file set to a different remote system.
n this example, cache system 1 is initially configured as single-writer and cache system 2 is
initially configured as read-only. Perform the following steps in the sequence described to
reverse the roles so that system 2 becomes single-writer and system 1 becomes read-only:
On cIient system 1:
1. Use the -- command to verify that Remote Fileset Status and
Cache-Gateway Status are active, and that queue length is zero, as in Example 4-60:
Example 4-60 lswcachestate command
-- - --
- -- - - -
-- -
- - -
2. Unmount the file system that contains the cache file set to be changed.
3. Use the command to change the mode of the remote file set to read-only, as in
Example 4-61:
Example 4-61 chwcache command
- -
Tip: You cannot change the mode of a client cache file set from Local Update mode to any
other mode. You can change the mode of a client cache file set from read-only to
single-writer and vice versa, and from both read-only and single-writer to Local Update.
Note: The command unlinks the file set, so any application attempt to access the
file set results in an error. Best practice is to remove application access to the file set
before submitting the command.
The command fails if there are pending updates that have not yet been
synchronized. n this case, create a file in cache. This triggers the system to perform a
recovery and resynchronization. Then resubmit the command.
On the home system:
1. Use the - command to change system 1 to read-only, as in Example 4-62.
Example 4-62 chwcachesource command
- - -- --
- - -- --
2. Use the - command to change system 2 to single writer (rw) mode, as in
the following example:
On cIient system 2:
1. Use the command to ensure that all of the files in the file system are cached
and current on client system 2, as in Example 4-63.
Example 4-63 runpreop command
- -
2. Unmount the file system that contains the cache file set to be changed.
3. Use the command to change the mode of the remote file set to single writer (sw),
as in Example 4-64:
Example 4-64 chwcache command
- - -
4.7.9 Move home system of a singIe-writer mode fiIe set to a different system
You can use the --remotepath and --homeip options of the command to move the
home file set of a single-writer mode client cache file set to a different system. This might be
useful if a home system becomes unavailable.
Tip: policyName is the name of a policy that was created with the command
using the -R option to create a set of rules that fetch all of the files in the file set when
invoked by the command.
Note:The command unlinks the file set, so any application attempt to access the
file set results in an error. Best practice is to remove application access to the file set
before submitting the command.
Tip: You cannot change the mode of a client cache file set from local-updates mode to any
other mode. You can change the mode of a client cache file set from read-only to
single-writer and vice versa, and from both read-only and single-writer to local-updates.
Restrictions
The following restrictions apply:
1. The client cache file set must have complete and current data.
2. The new home file set cannot contain any data.
3. You can only move the home system of a client cache file set that is identical in content
and size to its corresponding file set on the home system. Because a local-updates mode
file set by design can contain changes that are not pushed to the home site, the
CL command fails if you attempt to change the home P and remote path of a
local-updates mode file set. Similarly, because it is likely that a read-only mode file set is
not identical to its home file set, the CL command fails if you attempt to change
the home P and remote path of a read-only mode file set while it is in read-only mode. To
change the home P and remote path of a read-only mode client cache file set, you must
first convert its mode to single-writer mode. See Moving home system of a read-only
mode file set to a different system on page 228. Using the --remotepath and --homeip
options of the chwcache command performs a policy scan to identify all of the files in the
specified client cache file set and queues operations to push all of its data to the new
home as specified by the --remotepath and --homeip options.
Considerations
Observe the following considerations:
All applications that access the file system must be stopped before the operation is
started, and must remain stopped until the operation completes.
The entire file system is suspended while the operation is in progress, so all of the client
cache file sets in the file system are affected.
The operation takes a considerable amount of time proportional to the number of files in
the file set, because it scans the file set. During the scan, the operation must not be
stopped prematurely. f the operation does not complete, the file system must be
remounted and the operation must be restarted.
The operation is asynchronous. The operation is complete when the queue is empty.
Moving home system of a read-onIy mode fiIe set to a different system
f the client cache file set is read-only mode, its not guaranteed to have current and complete
data synchronized with the home file set, and therefore the CL command fails if you
attempt to change the home P and remote path of a read-only mode file set while it is in
read-only mode. Submit the CL command and ensure that the client cache file set
has current and complete data synchronized with the home cache file set before you convert
the read-only mode client cache file set to single-writer mode and push data to the new home.
Once the home file set and client cache file set now in single-writer mode are fully
synchronized, the client cache file set can be converted back to read-only mode
Wide Area Network caching Iimitations
A caching file set is a GPFS file set with additional WAN caching attributes. Keep the following
limitations in mind when using the WAN caching function:
The SONAS systems at the home and cache locations must have a SONAS code version
of at least 1.3.0 installed.
You cannot link any file set, whether dependent or independent, into a cache file set.
When WAN caching is recovering from a WAN caching failure, all WAN caching
application requests are blocked.
WAN caching caches all file data, extended attributes and access control lists (ACLs) from
the home site to the cache site. However, WAN caching does not cache any metadata
such as snapshots, file sets or quotas, and it does not perform D mapping between the
home site and the cache site.
t is the customer's responsibility to ensure replication between LDAP servers used by the
home and cache system.
When using Active Directory with NS, SONAS does not support CFS users that are not
configured in NS.
SAMBA PDC / N4 authentication is not supported.
WAN caching is performed only at the file set level. You can selectively prevent certain
files from being cached by using a standard GPFS policy rule.
Data transfer rates are not guaranteed.
The cache eviction function requires that quotas are enabled at the cache level.
Only whole files are cached. Even if a read is requested for a partial file, caching brings
the entire file into the cache. n the case of a write request, only the changed bytes are
pushed from the cache site to the home site in normal scenarios. n the case of certain
caching failures, where a file system recovery is performed, WAN caching recovers the
entire file, not just the changed bytes.
WAN caching does not support file locking across systems. f an application locks a
cached file at the cache system only the cached file is locked; the corresponding file at the
home system is not locked by this request.
WAN caching does not support NFS delegation.
f there is a file clone at the home system, it is not displayed as a clone in the cache
system; instead, it displays as a separate file. Likewise, if a file is cloned in the cache
system, the clone is not replicated to the home system.
Snapshots taken at the home system are not in cache. Use the snapdir variable for
specifying snapshot location because the .snapshots reserved directory is the same at
both the cache and home systems.
The hard links at the home site are not copied as hard links to cache. f a hard link is
created in cache, that relationship is also created at the home site.
f a file is renamed at the home site, the file in cache is deleted and created again in
cache. This results in the file being assigned a different inode number at the cache site.
The clone, snapshot, hard link and rename restrictions also apply to data that is
prefetched from the home site to the cache site.
Special files like device files and sockets are not replicated between the cache and home
sites.
There is no option to encrypt or compress WAN caching data.
4.8 WAN caching creation and configuration waIkthrough
n this section, we show the steps required to set up WAN caching, using the SONAS CL.
The environment is shown in Figure 4-24.
Figure 4-24 WAN caching, using the SONAS CLI
Configuring the home site cIuster
Follow these steps:
1. Log in to the home SONAS cluster CL.
2. Create and link a new file set using the - and - commands (see
Example 4-65). f you have and existing linked fileset, skip this step and continue with
step 4.
Example 4-65 Create and link a new file set
-- - -
- ---
---
-- - - -
-
- ---
---
3. Create a new share using the CL command (see Example 4-66). f you have an
existing share, skip this step and continue with step 4.
Example 4-66 Create a share
-- - -
--- - -
- ---
---
4. Check the exports using the - command (see Example 4-67).
Example 4-67 List exports
-- -
-
-
-
---
5. Create a caching enabled share using the - command (see Example 4-68).
Example 4-68 Create caching enabled share
-- - -
---
6. Use the -- command to list and check the share (see Example 4-69).
Example 4-69 list cache sources
-- --
- -
-- -
- -
---
7. Use the -- --details command for a detailed list (see Example 4-70).
Example 4-70 List detailed cache sources
-- -- -
-
-
-
----
------
-----
-
---
Configuring the cIient site cIuster
Follow these steps:
1. Log in to the home SONAS cluster CL.
2. Add selected node to the cachenode list using the command (see
Example 4-71).
Example 4-71 Add node to caching nodes
-- - -
---
3. List node using the - -v command (see Example 4-72).
Example 4-72 List Nodes
-- -
- -
- -- -- -- - - -
-- - - -
-- -
-
- -
- - -

-
-
-

- --
-
-

-- -
-

-- -
-

---
4. Use the command to create a wan-caching fileset on the client cluster (see
Example 4-73).
Example 4-73 Create caching fileset on client
-- - - -
-
---
5. List the wan-caching file sets of the client cluster using the - command (see
Example 4-74).
Example 4-74 List Cache file sets
-- - -
-
-
- -
-
---
6. Use the -- --connectionstatus command to check the connection between the
home and the client cluster (see Example 4-75).
Example 4-75 List chache connection status
-- -- - - --
-
- -- - - -
-- -
- - -
-
---
7. Create an export on the chached using the command (Example 4-76).
Example 4-76 Create an export using the mkexport command
-- -
- - --- -
---
8. List exports using the - command (see Example 4-77).
Example 4-77 List exports
-- -
-
-
-
-
-
-
---
Copyright BM Corp. 2012. All rights reserved. 235
Chapter 5. Backup and recovery, avaiIabiIity,
and resiIiency functions
n this chapter, we illustrate SONAS components and external products that can be used to
guarantee data availability and resiliency. We also provide details of the Tivoli Storage
Manager integration.
We describe the following topics:
Backup and recovery of files in a SONAS cluster
Configuring SONAS to use HSM
Replication of SONAS data
SONAS Snapshots
Disaster Recovery
5

SoNAS ACE

Caricato da

Informazioni sul documento

Copyright

Formati disponibili

Condividi questo documento

Condividi o incorpora il documento

Opzioni di condivisione

Hai trovato utile questo documento?

Questo contenuto è inappropriato?

Copyright:

Formati disponibili

SoNAS ACE

Caricato da

Copyright:

Formati disponibili

Copyright BM Corp. 2012. All rights reserved.

226 SONAS mplementation and Best Practices Guide

Potrebbero piacerti anche