Pattern Sets and Data Sets

2015-01-16 04:38:56 UTC

2015 Citrix Systems, Inc. All rights reserved. Terms of Use | Trademarks | Privacy Statement

Contents

Pattern Sets and Data Sets ..................................................................................

How String Matching works with Pattern Sets and Data Sets..............................

Configuring a Pattern Set .......................................................................

Configuring a Data Set...........................................................................

Using Pattern Sets and Data Sets ..............................................................

11

Sample Usage .....................................................................................

13

Pattern Sets and Data Sets

Policy expressions for string matching operations on a large set of string patterns tend to
become long and complex. Resources consumed by the evaluation of such complex
expressions are significant in terms of processing cycles, memory, and configuration size.
You can create simpler, less resource-intensive expressions by using pattern matching.
Depending on the type of patterns that you want to match, you can use one of the following
features to implement pattern matching:

A pattern set is an array of indexed patterns used for string matching during default
syntax policy evaluation. Example of a pattern set: imagetypes {svg, bmp, png, gif, tiff,
jpg}.

A data set is a specialized form of pattern set. It is an array of patterns of types

number (integer), IPv4 address, or IPv6 address.

In many cases, you can use either pattern sets or data sets. However, in cases where you
want specific matches for numerical data or IPv4 and IPv6 addresses, you must use data
sets.
Note: Pattern sets and data sets can be used only in default syntax policies.
To use pattern sets or data sets, first create the pattern set or data set and bind patterns
to it. Then, when you configure a policy for comparing a string in a packet, use an
appropriate operator and pass the name of the pattern set or data set as an argument.

How String Matching works with Pattern

Sets and Data Sets
A pattern set or data set contains a set of patterns, and each pattern is assigned a unique
index. When a policy is applied to a packet, an expression identifies a string to be
evaluated, and the operator compares the string to the patterns defined in the pattern set
or data set until a match is found or all patterns have been compared. Then, depending on
its function, the operator returns either a boolean value that indicates whether or not a
matching pattern was found or the index of the pattern that matches the string.
Note: This topic explains the working of a pattern set. Data sets work the same way. The
only difference between pattern sets and data sets is the type of patterns defined in the
set.
Consider the following use case to understand how patterns can be used for string
matching.
You want to determine whether the URL suffix (target text) contains any of the image file
extensions. Without using pattern sets, you would have to define a complex expression, as
follows:

HTTP.REQ.URL.SUFFIX.CONTAINS("svg") HTTP.REQ.URL.SUFFIX.CONTAINS("bmp") HTTP.REQ.URL.SUFFIX

HTTP.REQ.URL.SUFFIX.CONTAINS("gif") HTTP.REQ.URL.SUFFIX.CONTAINS("tiff") HTTP.REQ.URL.SUFFIX.C
If the URL has a suffix of "jpg," with the above compound expression, the NetScaler
appliance has to iterate through the entire compound expression sequentially, from one sub
expression to the next, to determine that the request refers to a jpg image. The following
figure shows the steps in the process.

How String Matching works with Pattern Sets and Data Sets
When a compound expression includes hundreds of sub expressions, the above process is
resource intensive. A better alternative is an expression that invokes a pattern set, as
shown in the following figure.

During policy evaluation as shown above, the operator (CONTAINS_ANY) compares the string
identified in the request with the patterns defined in the pattern set until a match is found.
With the Sample_Patset expression, the multiple iterations through six sub expressions
are reduced to just one.
By eliminating the need to configure compound expressions that perform string matching
with multiple OR operations, pattern sets or data sets simplify configuration and accelerate
processing of requests and responses.

Configuring a Pattern Set

To configure a pattern set, you must specify the strings that are to serve as patterns. You
can manually assign a unique index value to each of these patterns, or you can allow the
index values to be assigned automatically.
Note: Pattern sets are case sensitive (unless you specify the expression to ignore case).
Therefore, the string pattern "product1," for example, is not the same as the string
pattern "Product1."
Points to remember about index values

You cannot bind the same index value to more than one pattern.

An automatically assigned index value is one number larger than the highest index value
of the existing patterns within the pattern set. For example, if the highest index value
of existing patterns in a pattern set is 104, the next automatically assigned index value
will be 105.

If you do not specify an index for the first pattern, index value 1 is automatically
assigned to that pattern.

Index values are not regenerated automatically if one or more patterns are deleted or
modified. For example, if the set contains five patterns, with indexes from 1 through 5,
and if the pattern with an index of 3 is deleted, the other index values in the pattern
set are not automatically regenerated to produce values from 1 through 4.

The maximum index value that can be assigned to a pattern is 4294967290. If that value
is already assigned to a pattern in the set, you must manually assign index values to any
newly added patterns. An unused index value that is lower than a currently used value
cannot be assigned automatically.

To configure a pattern set by using the command line

interface
At the command prompt, do the following:
1. Create a pattern set.
add policy patset <name>
Example:
> add policy patset samplepatset
2. Bind patterns to the pattern set.
bind policy patset <name> <string> [-index <positive_integer>]

Configuring a Pattern Set

Example:
> bind policy patset samplepatset product1 -index 1
Note: Repeat this step for all the patterns you want to bind to the pattern set.
3. Verify the configuration.
show policy patset <name>

To configure a pattern set by using the configuration

utility
1. Navigate to AppExpert > Pattern Sets.
2. In the details pane, click Add to open the Create Pattern Set dialog box.
3. Specify a name for the pattern set in the Name text box.
4. Under Specify Pattern, type the first pattern and, optionally, specify values for the
following parameters:

Treat back slash as escape characterSelect this check box to specify that any
backslash characters that you might include in the pattern are to be treated as
escape characters.

IndexA user assigned index value, from 1 through 4294967290.

5. Verify that you have entered the correct characters, and then click Add.

6. Repeat steps 4 and 5 to add additional patterns, and then click Create.

Parameter Descriptions (of commands listed in the

CLI procedure)
add policy patset
No parameters provided in this topic or the command has no parameters. View
description(s) in command reference Top

bind policy patset

No parameters provided in this topic or the command has no parameters. View
description(s) in command reference Top

show policy patset

name

Configuring a Pattern Set

Name of the pattern set for which to display the detailed information. If a name is not
provided, a list of all pattern sets configured on the appliance is shown.
View description(s) in command reference Top

Configuring a Data Set

To configure a data set, you must specify the strings that are to serve as patterns, and
assign a type (number, IPv4 address, or IPv6 address) to each pattern. You can manually
assign a unique index value to each of these patterns, or you can allow the index values to
be assigned automatically.
Note: Data sets are case sensitive (unless you specify the expression to ignore case).
Therefore, the string pattern "product1," for example, is not the same as the string
pattern "Product1."
The rules applied for index values of data sets are the same as those applied for pattern
sets. For information about index values, see "Configuring a Pattern Set."

To configure a data set by using the command line

interface
At the command prompt, do the following:
1. Create a data set.
add policy dataset <name> <type>
Example:
> add policy dataset sampledataset ipv4
2. Bind patterns to the data set.
bind policy dataset <name> <value> [-index <positive_integer>]
Example:
> bind policy dataset sampledataset 10.102.29.1 -index 1
Note: Repeat this step for all the patterns you want to bind to the data set.
3. Verify the configuration.
show policy dataset <name>

To configure a data set by using the configuration

utility
Navigate to AppExpert > Data Sets, click Add and specify the relevant details.

Configuring a Data Set

Parameter Descriptions (of commands listed in the

CLI procedure)
add policy dataset
No parameters provided in this topic or the command has no parameters. View
description(s) in command reference Top

bind policy dataset

No parameters provided in this topic or the command has no parameters. View
description(s) in command reference Top

show policy dataset

No parameters provided in this topic or the command has no parameters. View
description(s) in command reference Top

10

Using Pattern Sets and Data Sets

Default syntax policy expressions that take pattern sets or data sets as an argument can be
used to perform string matching operations.
The usage is as follows:
<text>.<operator>("<name>")
where,

<text> is the expression that identifies a string in a packet. Example:

HTTP.REQ.HEADER("Host").

<operator> is one of the operators described in the following table.

Table 1. Operators for pattern sets and data sets

11

Operator

Description

<text>.CONTAINS_ANY(<name>)

Returns true if the target text contains

one or more of the patterns defined in
the specified pattern set or data set.

<text>.SUBSTR_ANY(<name>)

Returns the first string that matches

any pattern defined in the specified
pattern set or data set.

<text>.BEFORE_STR_ANY(<name>)

Returns the text that is present before

the first occurrence of any of the
patterns defined in the specified
pattern set or data set.

<text>.AFTER_STR_ANY(<name>)

Returns the text that is present after

the first occurrence of any of the
patterns defined in the specified
pattern set or data set.

<text>.EQUALS_ANY (<name>)

Returns true if the target text exactly

matches any of the patterns defined in
the specified pattern set or data set.

<text>.ENDSWITH_ANY(<name>)

Returns true if the target text ends

with any of the patterns that are
defined in the specified pattern set or
data set.

<text>.STARTSWITH_ANY(<name>)

Returns true if the target text starts

with any of the patterns that are
defined in the specified pattern set or
data set.

Using Pattern Sets and Data Sets

<text>.STARTSWITH_INDEX(<name>)

Evaluates whether the target text

starts with any of the patterns that
are defined in the specified pattern
set or data set. If a match is found,
the index of the matching pattern is
returned. Otherwise, 0 is returned.

<text>.ENDSWITH_INDEX(<name>)

Evaluates whether the target text

ends with any of the patterns that are
defined in the specified pattern set or
data set. If a match is found, the
index of the matching pattern is
returned. Otherwise, 0 is returned.

<text>.CONTAINS_INDEX(<name>)

Evaluates whether the target text

contains any of the patterns that are
defined in the specified pattern set or
data set. If a match is found, the
index of the matching pattern is
returned. Otherwise, 0 is returned.

Evaluates whether the target text

exactly matches any of the patterns
that are defined in the specified
pattern set or data set. If an exact
match is found, the index of the
pattern is returned. Otherwise, 0 is
returned.
<name> is the name of the pattern set or data set
<text>.EQUALS_INDEX(<name>)

For sample usage, see "Sample Usage."

12

Sample Usage
To understand the usage of pattern sets in expressions, consider the example of a pattern
set named "imagetypes."
Table 1. Pattern set "imagetypes"
Patterns

Index value

svg

bmp

png

gif

tiff

jpg
6
Example 1: Determine whether the suffix of an HTTP request is one of the file extensions
defined in the "imagetypes" pattern set.

Expression. HTTP.REQ.URL.SUFFIX.EQUALS_ANY("imagetypes")

Sample URL. http://www.example.com/homepageicon.jpg

Result. TRUE

Example 2: Determine whether the suffix of an HTTP request is one of the file extensions
defined in the "imagetypes" pattern set, and return the index of that pattern.

Expression. HTTP.REQ.URL.SUFFIX.EQUALS_INDEX("imagetypes")

Sample URL. http://www.example.com/mylogo.gif

Result. 4 (The index value of the pattern "gif".)

Example 3: Use the index value of a pattern to determine whether the URL suffix is within
a specified index-value range.

Expression. HTTP.REQ.URL.SUFFIX.EQUALS_INDEX("imagetypes").GE(3) &&

HTTP.REQ.URL.SUFFIX.EQUALS_INDEX("imagetypes").LE(5)

Sample URL. http://www.example.com/mylogo.gif

Result. TRUE (The index value of gif file types is 4.)

Example 4: Implement one set of policies for file extensions bmp, jpg, and png, and a
different set of policies for gif, tiff, and svg files.
An expression that returns the index of a matched pattern can be used to define traffic
subsets for a web application. The following two expressions could be used in content
switching policies for a content switching virtual server:
13

Sample Usage

14

HTTP.REQ.URL.SUFFIX.EQUALS_INDEX("imagetypes").LE(3)

HTTP.REQ.URL.SUFFIX.EQUALS_INDEX("imagetypes").GE(4)