Support Vector Machines

Support Vector Machines, a promising new method for

the classification of both linear and nonlinear data. A
support vector machine (or SVM) is an algorithm that works
as follows. It uses a nonlinear mapping to transform the
original training data into a higher dimension. Within this
new dimension, it searches for the linear optimal separating
hyperplane(that is, a “decision boundary” separating the
tuples of one class from another). Data from two classes can
always be separated by a hyperplane. The SVM finds this
hyperplane using support vectors (“essential” training
tuples) and margins (defined by the support vectors). SVMs
can be used for prediction as well as classification. They
have been applied to a number of areas including
handwritten digit recognition, object recognition, and
speaker identification, as well as benchmark time-series
prediction tests.

The Case When the Data Are Linearly Separable

Let's first look at the simplest case a two-class problem where the
classes are linearly separable. Let the data set D be given as (X1,
y1), (X2, y2), : : : , (X |D|, y |D|),where Xi is the set of training
tuples with associated class labels, yi. Each yican
 take one of two values, either+1 or -1 (i.e., yi € {+1, -1}),
corresponding to the classes buys computer = yes and buys
computer = no, respectively.
Let's consider an example based on two input attributes, A1 and
A2, as shown in Figure below from the graph, we see that the 2-D
data are linearly separable because a straight line can be drawn to
separate all of the tuples of class+1 from all of the tuples of class
-1.
There are an infinite number of separating lines that could
be drawn. We want to find the “best” one, that is, one that will
have the minimum classification error on previously unseen
tuples. An SVM approaches this problem by searching for the
maximum marginal hyperplane.