Self – Organizing Neural networks

Kohonen Self organizing Feature Maps

(SOM)

S. Vivekanandan

Cabin: TT 319A
E-Mail: svivekanandan@vit.ac.in
Mobile: 8124274447
 Self Organizing Maps (SOM)

• Learning is based only upon the input data and is independent

of the desired output – no error is calculated – Unsupervised
learning.
• Net may respond to several output categories on training, but
only one of the several neurons has to respond.
• Mechanism by which only one unit is chosen to respond is
called competition.
• Principle is Winner-Takes all. (only one neuron in the
competing group will have a non-zero output signal when the
competition is completed.)
• A common use for unsupervised learning is clustering based
neural networks.
30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 2
• In a clustering net, there are as many inputs as an input vector
components.
• The weight vector for an output unit in a clustering net is
called exemplar or code-book vector – which the net has
placed on that cluster.
• During training, the net determines the output unit that is the
best match for the current input vector; the weight vector for
the winner is then adjusted wrt the nets learning algorithm.
• Updation of weight is by forming a new weight vector that is a
linear combination of the old weight vector and the current
input vector

30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 3

 Methods used for determining the winner

• Method 1
Uses the square Euclidean distance between the input vector and
the weight vector, and choses the unit weight vector has the
smallest distance from the input vector

Method 2
Uses the dot product of the input vector and the weight vector
which is the net input to the corresponding cluster unit.
The largest dot product corresponds to the smallest angle between
the input and the weight .

30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 4

 Architecture of SOM

30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 5

 Kohonen Self organizing maps

• Kohonen introduced it
• Also termed as topology- preserving maps.
• Units located physically next to each other will respond to
classes of input vectors that are likewise located to each other.
• Groups the input data in to clusters which are commonly used
in unsupervised training.
• All the units in the neighbourhood that receive positive
feedback from the winning unit participate in the learning
process.

30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 6

 Kohonen Self organizing maps

• Architecture is similar to competitive net.

• Training should take place over an extended region of the
network centered on the maximally active mode.
• Hence the concept of ‘neighbourhood’ should be defined for
the net.
• This may be fixed by the spatial relation between nodes within
the self organizing layer
• Two neighbourhood arrays in the form of rectangular and
hexagonal grids are shown.
• In all cases, neighbourhood are shown delimited wrt a shaded
unit at distances of 1,2 and 3 arrays from this node.

30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 7

 Kohonen Self organizing maps

• The rectangular and hexagonal arrays have 25 and 19 nodes

respectively in their distance.
• Although three dimensional arrays of nodes are conceivable,
they are not used owing to their complexity.
• The competitive network is similar to a single-layered feed
forward network except that there are connection, usually
negative, between the output nodes.
• because of these connections the output nodes tend to
compete to represent the current input pattern.
• The winning unit is indicated by # and the other units are
indicated by *.

30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 8

 Neighbourhood scheme for SOM

30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 9

 Algorithm

30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 10

30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 11
30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 12
30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 13
30-03-2017 Dr. S. Vivekanandan Asso. Prof.-SELECT 14