VR Siddhartha Engineering College

TechnoFest-2009
Computer Science &Engineering

A Paper presentation on

KOHONEN NEURAL NETWORKS

Presented by:

Vuyyuru Deepu Kumar Jakka Santhosh

06A81A0567 06A81A0576
III/IV BTech, CSE III/IV BTech, CSE
vuyyurudeepukumar@gmail.com santhoshmay24@gmail.com
Ph: 9966454405 Ph: 9490784744

SRI VASAVI ENGINEERING COLLEGE

TADEPALLIGUDEM, WEST GODAVARI DT.
 When the only tool you have is a hammer, every problem you
encounter tends to resemble a nail

ABSTRACT:

Neural network has taken its importance since 1950s

from the invention of computer. For many centuries, one of the goals of
human mankind has been to develop machines we envision these
machines as performing all cumbersome and tedious tasks so that we
might enjoy a more fruitful life. As we see that people and animals
recognition of patterns and images is much faster than most advanced
computers though computers outperform both biological and artificial
neural systems for tasks based on precise and fast arithmetic operations,
artificial neural systems represent the promising new generation of
information processing networks. Over the past fifteen years, a view has
emerged that computing based on models inspired by our understanding
of the structure and function of the biological neural networks may hold
the key to the success of solving intelligent tasks by machines the new
field is called “Artificial neural networks or Neural networks”.

CONTENTS

1. INTRODUCTION
2. ELEMENTARY NEURO PHYSIOLOGY
3. KOHENON NETWORKS
4. BENEFITS OF NEURAL NETWORK
5. FUTURE TRENDS IN NEURAL NETWORKS
1. INTRODUCTION
The only tool we have is a
sequential computer, and then
every problem we encounter will
be cast in terms of a sequential
algorithm.
Every Computer designers,
engineers, and programmers, all of whom
are striving to create more “intelligent
“computer systems think ………………
…..!
Why can’t we build a computer that
thinks? Why can’t we except machines that
can perform 100 million floating-point
calculations per second to be able to
comprehend the meaning of shapes in
visual images, or even to distinguish
between different kinds of similar objects?
Why can’t that same machine learn from
experience, rather than repeating forever an
explicit of instructions generated by a
human programmer?
There are many applications that
we would like to automate ,but have not
automated due to complexities associated
with programming a computer to perform
the tasks to a large extent these tasks are
not possible for programming as they pose
many oppositions .
Let us take a small example shown
by picture below. How is that we can see
the dog in the image quickly, yet a
computer cannot visualize the
discrimination as we do it.
This question is especially poignant
when we considered that the switching time
of the components in modern electronic
computers are more than seven orders of
magnitude faster than the cells that
comprise our neurobiological system. This
is partially answered by the fact that the
human brain is a complex built system
when compared to the computer.
In many real world applications, we
want our computers to perform complex
pattern recognition such as the one just
described. Since our conventional
computers are obviously not suited to this
type of problem, we therefore borrow
features from the physiology of the brain as
the basis for our new processing models.
This technology has
Come to be known as artificial neural
system or simply neural networks.
We begin defining a neural network
structure as a collection of parallel
processors connected together in the form
of a direct graph. By referring, a figure as
atypical
Network diagram we can schematically
represents each processing element as a
node
with connections between units indicated
by arcs. In this example we will restrict the
Number of characters the neural network
must recognize to 10 decimal digits, 0,
1,2……9.Our objective is to have the
neural network determine which of the 10
digits a particular Hand drawn character is,
we can create a net work structure that has
10 discrete outputs One for each character
to be identified .if we insist the output units
behave according to A simple on –off
strategy in the process of converting an
input signal to output signal becomes a
simple majority function.
The schematic diagram represents the
character-recognition problem described in
the text. In this example, application of an
input pattern on the bottom layers of
processors can cause many of the second
layer, or hidden layer, units to activate. The
activity on the hidden layer should then
cause exactly one of the output layers to
activate-the one associated with the pattern
being identified. We can also observe the
large number of connections required for
the small network.
Based on the conditions,
we can arbitrarily choose the pixel image as
a 10*8matrix, using a 1 to represent the
pixel that is “on” and a “0” to represent a
pixel that is “off”. At this point, all that
remains is to size the number of processing
units (called hidden units) that must be used
internally, to connect them to the input and
output units called weight connections.

2. ELEMENTARY
NEUROPHYSIOLOGY
There are many basic concepts that
Simple pictorial representation of the neuron
have a more universal significance .In this
regard we look first at individual neurons
then at the synaptic junctions between
neurons.
2.1. SINGLE NEURON PHYSIOLOGY:
The features of biological neural
networks are attributed to its structure and
its function. The fundamental unit of the
network is called a neuron or a nerve cell.
The major components of a typical nerve
cell in the central nervous system are as
follows
• Cell body;
• Axon
• Nucleus
• Dendrites
• Synapse
• Myelin sheath
The neuron which consists of a cell body or
soma where the cell nucleus is located tree
like nerve fibers called dendrites are The transmission of a signal one cell
associated with cell body they connect one to another at a synapse is a complex
nerve chemical process in which specific
Cell to the other and also use full in transmitter ionic compounds transmit
transmitting one neuron to the other. A signals when these ionic
single long Compounds are released. The effect is to
Fiber called axon extends from the cell rise or lower electric potential inside the
body which eventually branches in to body of the receiving cell. if this signal
stands and reaches a threshold, an electrical activity in
Sub stands connecting to many other the form of
neurons at the synaptic junction or the Short pulses are generated. When this
synapse. happens the cell is said to be fired.
Generally the electrically activity is
confined to the interior of the neuron where
as the chemical mechanism operates at the
synapse. Dendrites serve as receptors for
signals from other
neurons. The purpose of the axon is to
transmit the generated neural activity to
other nerve cell or to other muscle fiber
neuron.

2.1.1Process:
In the state of inactivity the
interior of the neuron, the protoplasm is
negatively charged against the surrounding
neural liquid containing positive sodium
ions (Na+). The Resulting resting potential
of about -70mv is supported by the action
of the cell membrane which is impenetrable
for the sodium (Na+) ions. This causes a
deficiency of +ve ions in the protoplasm. The presynaptic terminals.
Signals arriving from the synaptic junctions
may result in Temporary depolarization of
the resting potential. When the potential is
increased -60mv, the membrane loses its
impermeability against the sodium (Na+)
ion which enters the protoplasm and
reduces the potential difference.

This sudden change in membrane

potential causes the neuron to discharge
then the neuron is said to be fired. The
intensity of the signal is encoded in the
frequency of sequence of pulses of activity,
which can Range from about 1 to 100/sec.
if the induced polarization potential is
positive the synapse is termed excitatory
because the influence of the synapse tends
to activate the post synaptic neuron. If the
polarization potential is negative the
synapse is called Inhibitory, since its
contracts excitation of the neuron. All the
synaptic endings of an Axon are either of an
excitatory or an inhibitory nature
The process of transmission
of the signal in a neuron.
The cell body of a neuron acts as a kind of
summing device due to the net depolarizing
effect of its signals this net effects decay
with time constants of 5 to 10 milli seconds
When the total magnitude of the
depolarization potential in the cell body
exceeds the critical threshold (about 10mv),
the neuron fires. The activity of the given
synapse depends on the rate of arriving
signals. An active synapse, which
repeatedly triggers activation of its post
synaptic neuron, will grow in strength,
while others will gradually will weaken
thus the strength of the synaptic connection
gets modified continuously .This
mechanism of synaptic plasticity in
structure of neural connectivity, known As
Hebb’s rule, appears to play a dominant
role in the complex process of learning
 A Kohenon net has the special
property that it does its best to preserve
continuity. That is, if two colors are nearby
each other in the input space, they will be
nearby each other in output space. In fact,
this continuous mapping turns out to be
simply the topological stretching of a figure
in n-space into a corresponding figure in
output space.

Neurotransmission. Of course, the network isn't

perfectly successful at this task, for a
3. KOHENON NETWORK number of reasons. First, some figures
simply cannot be mapped continuously into
2-space. Consider, for example, the solid
3.1 PATTERN RECOGNITION BY
cube: it just can't be flattened into the plane.
KOHENON NETWORK:
Secondly, the network is constructed of a
collection of discrete neurons, rather than a
A Kohenon Network is a
continuous surface. Therefore, sometimes
breed of neural network designed to group
continuity appears to get lost: If one color
similar inputs together by having them
region gets stretched into a long blob on the
represented by nearby neurons in a neural
neuron surface, then two nearby colors may
network. After the net is trained, each
appear distant on the map. There is still a
neuron represents a particular input feature
continuous transition between them, but it
vector, and domain of inputs has been
is lost in the discrete steps of the neurons.
organized spatially across the field of
I.e., Kohonen nets preserve continuity, but
neurons in a way that reflects the original
they don't care much about preserving
organization of the input data.
distance.
One way to visualize this process is
Finally, Kohenon nets go about
as follows. The input is a set of feature
constructing this mapping in a fairly
vectors in n-space. These vectors will be
haphazard way, examining only one input
mapped onto the field of neurons in m
vector at a time. In light of this, they do
dimensions. A particular feature vector is
amazingly well at their task. With badly
mapped onto the neuron which most closely
tuned parameters, they construct rather poor
resembles it; in this way, we can represent
mappings; even with good tuning, the net
all possible inputs with a finite number of
often builds its way into a corner, causing it
neurons. We can think of the output of the
to have to break continuity for some
net being the discrete vector ``address'' of
regions.
the winning neuron.

In my program, input vectors are

confined to the unit cube, and each vector is
displayed as an RGB triplet color. The
neural net is a flat surface (2-space), so
m=2. It's certainly conceivable to construct
neural surfaces of greater dimension, but
two dimensions illustrate the point quite
handily, and are easy to display and
visualize.

Left: the edges of the unit cube, as
colored in RGB space. Right: A correct
mapping into 2-space which preserves
continuity.
Figure 2. Left: A frustrated Kohonen
network can't quite continuously connect
two of the original edges. Right: A correct
solution discovered by the network relies
on the ``wrap-around'' shape of toroidal 2-
space. The arrows indicate where the figure
connects with itself.
Overall, however, self-ordering
maps are surprisingly successful. Even
humans have to think a little bit to
discover how the edges of a cube can be
mapped into 2-space. When the Kohonen
net solved this problem, it almost always
preserved the continuity of at least ten
edges, and once even came up with an
unexpected but correct solution which takes
advantage of toroidal 2-space (from the
above figure)
3.2 Different types of kohenon
networks
3.2.1. Linear Vector Quantization
Each competitive unit
corresponds to a cluster, the centre of which
is called ‘the code book vector’ (also
‘The Focus’). When an input is presented to
the network, the codebook vector nearest
the input is found and the code book vector
is then alerted to be more like the input.
The codebook vector is moved a proportion
of the distance between the input and the
codebook vector. This is very similar to
macqueen’s k-means algorithm [21] except
in this case, the distance moved is the
reciprocal of the number of cases that have
been assigned to the winning cluster. The
VQ algorithm is often used for off-line
learning where the training data is stored
and the training algorithm is passed over
the data many times.
3.2.2 Self organizing maps:
The kohonen self organizing map
(SOM) is a topological mapping from input
space. To the cluster of classes. This is the
most typical form of kohonen network and
is inspired by the way in which human
sensory impressions are neurologically
mapped. In to the brain such those spatial
or other relations among stimuli correspond
to the spatial relations between neurons. In
the SOM, the neurons are arranged in an n-
dimensional grid (where n is usually 2). As
long as the number of inputs is greater than
dimensionality of the input space, any
number of inputs can be used to train the
networks. a SOM attempts to arrange
clusters so that any two similar clusters are
spatially close . How ever two inputs that
are close to each other in the input space
are not always in clusters close to each
other. The SOM, performs data
compression, this means that high
dimensional data can be represented in a
much lower dimensional space. The SOM
is effectively a Smooth mapping between
input space and grid space.
ALGORITHM REPRESENTING THE
KOHONENE NETWORK:
A Kohonen net trains in the following
cycle:
Select an input vector.
Find the neuron which most closely
resembles it. (Winner takes all.)
Make that neuron even more like the input
vector. The alpha training constant (or not-
so-constant) determines how much this
neuron is changed. An alpha of 1.0 will
make this neuron exactly like the input
vector; an alpha of zero will not change it at
all.
Make all of his nearby neighbors more
closely resemble the input vector, as well.
The shape and size of the neighborhood are
also adjustable training parameters.
Repeat.
Notice how closely this process resembles
the Hair Care Training Process:
Lather.
Rinse.
Repeat.
In som, you have a wide variety of
options with regard to just how these steps
are performed. First, you may configure the
arrangement of the neurons. You may input
the desired dimensions of the map,
measured in neurons. A hexagonal
arrangement makes each neuron have six
equally near neighbors; a square (Cartesian)
arrangement gives each neuron four closest
neighbors. Hexagonal maps must have an
even number of rows. You may leave the
map a simple box figure with an outside
boundary, or close the figure into the
surface of a torus or donut. The map can be
initialized to either zero (black) neurons or
neurons representing a uniform random
distribution of inputs. The euclidean versus
taxicab option affects the measure of
distance used for feature space in step 2.
The neighborhood properties affect step 4.
You may input the initial radius of the
neighborhood. On Cartesian maps, the
shape of the neighborhoods may be either
square or circular. Neighborhood shape is
confined to a hexagon for hexagonal maps.
The ``Weighted'' option, if checked, causes
neighbor neurons modified in step 4 to be
affected less as a function of their distance
from the center of the neighborhood. The
winning neuron is modified according to
the full value of alpha as in step 3, and
more distant neurons are modified
according to alpha*f(distance), where f is a
normal bell curve. Without weighted
neighborhoods, every neuron in the
neighborhood is affected in the same
amount as the winning neuron.
The learning parameters include alpha,
alpha goal, and decay rate. As it happens,
training is most effective when alpha and
the neighborhood radius are allowed to
decrease with time. To accommodate this,
the program has a decay feature. Every
decay times through the training loop, the
neighborhood radius is decreased by one,
and alpha is moved closer to alpha goal.
The value of alpha is determined by
linearly interpolating between its initial
value and the goal, so that alpha arrives at
alpha goal just as the neighborhood radius
arrives at one. The training is stopped after
training with a radius of one, since not
many neurons would be affected in a
neighborhood of zero radius.
The map view is updated every vectors
times through the training loop when the
per vectors option is chosen. Updating the
view with each vector makes it convenient
to see the effects of varying the
neighborhood options, at the expense of
speed. Updating with each vector is
different from updating per 1 vectors in that
the each vector option only redraws those
neurons that appeared in the neighborhood
(to save a little time). Try watching the very
beginning of a training sequence in each
mode--you'll see directly how the weighting
and shape of neighborhoods affects
neurons.
The grid checkbox draws the inverse image
of the mapping as a grid surface in input
space. The window which displays input
vectors will have a grid surface
superimposed on it. Each node on the grid
corresponds to the value of a particular
neuron. This lets you directly visualize how
the map is stretching to fill feature space. I
recommend leaving the grid display off,
then turning it on to display a particular
grid of your choice; otherwise, the program
spends a lot of time redrawing.
All of the control parameters only affect the
initial settings of the map; they will not
change a map in the calculation process. To
invoke the changes, discarding the current
map, click Reset. Stop will pause the
training so the display can be inspected;
Continue will resume calculation without
reloading the parameters and initializing the
neurons. Read re-reads the input file; it is
functionally identical to Reset, but it lends a
nice symmetry to the display. Write stores
the map to the output file as previously
described. (When running in background
mode, the map is automatically written
after training ceases, and the program
terminates.) Quit is fairly self-explanatory. I
included a sentence on it for prosaic
closure, and because the capital Q's in this
typeface are very pretty.
4. BENEFITS OF NEURAL
NETWORKS.
It is apparent that a neural network
derives its computing power through, first,
its massively parallel distributed structure
and, second, its ability to learn and
therefore generalize; generalization refers
to the neural network producing reasonable
outputs for inputs not encountered during
training. These two information processing
capabilities make it possible for neural
network to solve complex (large scale)
problems that are currently intractable.
The use of neural networks offers
the following useful properties and
capabilities:
4.1 NON-LINEARITY:
A neuron is basically a non
linear device. A neural network made up of
interconnection of neurons is itself non
linear more over the non linearity is of a
special kind in the sense that it is
distributed through out the network. Non
linearity is highly important property
particularly if the underlying physical
mechanism responsible for
The generation of a input signal is
inherently non linear.
4.2 INPUT AND OUTPUT MAPPING:
A popular paradigm of learning
called supervised learning involves the
modification of synaptic weights of neural
networks by applying a label training
samples.The training of the network is
repeated for the many examples in the set
until the network reaches steady state.
Where there areno further significant
changes in the synaptic weights. Thus the
network learns from the examples by
constructing the input and output
Mapping
4.3 ADAPTIVITY:
Neural network has a built in
capability to adapt their synaptic weights to
changes in surroundings environment in
particular a neural network trained to
operate
In a specific environment conditions. More
over when it is operating in a non stationary
environment a neural network can be
designed to change its synaptic weights in
real time. The natural architecture of a
neural network for pattern classification,
signal processing, and control applications,
coupled with the adaptive capability of the
network make it a n ideal tool for use in
adaptive pattern classification, Adaptive
signal processing and adaptive control.
4.4 EVIDENTIAL RESPONSE:
In the context of pattern
classification a neural network can be
designed to provide information not only
about which particular pattern selects but
also about the confidence in the decision
made this latter information may be used to
reject ambiguous
Patterns and there by improving the
classification performance of the network

4.5 CONTEXTUAL INFORMATION:
Knowledge is represented by the
very structure and activation state of neural
network every neuron in the network is
potentially affected by the global activity
Of all other neurons in the network
consequently, contextual information is
dealt with naturally by a neural network.
4.6 FAULT TOLERANCE:
A neural network implemented in
the hard ware form has the potential to be
inherently fault tolerant in the sense that its
performance is degraded grace fully under
adverse operating conditions. How ever
owing to the distributed nature of
information in the network the damage has
to be extensive before the over all response
of the network is degraded seriously thus in
principle a neural network exhibits a
graceful degradation in performance rather
than catastrophic failure.
4.7 V.L.S.I IMPLEMENTABILITY:
The massively parallel nature of the
neural network makes it potentially fast for
the computation of certain tasks. This same
feature makes a neural network ideally
Suited for implementation using very large
scale integrated (V L S I) technology. The
particular virtue of VLSI is that it provides
a means of capturing truly complex
behavior
In highly hierarchical fashion which makes
it possible to use the neural network as a
tool
For real time applications involving pattern
recognition signal processing and control.
4.8 UNIFORMITY OF ANALYSIS AND
DESIGN:
Neural networks enjoy universality
as information processors it is in the sense
that the same notation is used in all the
domains involving the application of neural
Networks this feature manifests itself in
different ways
Neurons in one form or another represents
and ingredient common to all neural
networks. This common ability makes it
possible to share theories and learning
algorithms in different applications of
neural networks Modular networks can be
built through a seem less integration of
modules
4.9 NEURO BIOLOGICAL ANALOGY:
The design of the neural network is
motivated by analogy with the brain which
is a living proof that fault tolerant parallel
processing is not only physically possible
but also fast and power full? Neuro
biologists look to neural networks as
research tool for the interpretation of neuro
biological phenomena. On the other hand
engineers look to neuro
Biology for new ideas to solve problems
more complex than those based on
conventional hardwired designed
techniques. It provides a hope and belief
that physical under standing of neuro
biological structures could indeed influence
the art of electronics and VLSI
5. FUTURE TRENDS IN
NEURAL NETWORKS:
Neural networks and fuzzy logic
outline many applications. Most of the
applications fall in to few main categories.
According to the paradigms they are
literally thousands of applications of neural
networks and fuzzy logic in science and
technology and business with more and
more applications added as time goes on.
Thus neural network plays an important
role in the future of development of
mankind.
.