Ganesh Prabhakar P

ARTIFICIAL NEURAL NETWORKS

Abstract: This report is an introduction to Artificial Neural Networks. The various types of neural networks are explained and demonstrated. The connection between the artificial and the biological neural networks is also investigated and explained. The mathematical models involved are presented and demonstrated. The architecture of the Artificial Neural Networks is being explained in detail. Introduction: An Artificial Neural Network (ANN is an information processing paradigm that is inspired by the way biological nervous systems! such as the brain! process information. "# An artificial neural networ (ANN ! often $ust called a %neural network% (NN ! is a mathematical model or computational model based on biological neural networks. &t consists of an interconnected group of artificial neurons and processes information using a connectionist approach to computation. &n most cases an ANN is an adaptive system that changes its structure based on external or internal information that flows through the network during the learning phase.

A simple Neural Network 'roadly speaking! there are two methods for training an ANN! depending on the problem it must solve. A self(organi)ing ANN is exposed to large amounts of data and tends to discover patterns and relationships in that data. #esearchers often use this type to analy)e experimental data. A back(propagation ANN! conversely! is trained by humans to perform specific tasks. The true power and advantage of neural networks lies in their ability to represent both linear and non(linear relationships and in their ability to learn these relationships directly from the data being modeled. An ANN is configured for a specific application! such as pattern recognition or data classification! through a learning process. *earning in biological systems involves ad$ustments to the synaptic connections that exist between the neurons. Neural networks take a different approach to problem solving than that of conventional computers. +onventional computers use an algorithmic approach i.e. the computer follows a set of instructions in order to solve a problem. ,nless the specific steps that the computer needs to follow are known the computer cannot solve the problem. 'ut computers would be so much more useful if they could do things that we don-t exactly know how to do.

!" #u$an and Artificial Neurons % in&esti'atin' t(e si$ilarities

!") #ow t(e #u$an *rain Learns+
The structure of artificial neuron in a neural network is inspired by the concept of a biological neuron! which is shown in .igure /.0

The synapse +omponents of a neuron .igure /.0 'asically! it is a processing element (P1 in a brain2s nervous system that receives and combines signals from other similar neurons through thousands of input paths called dendrites. &f the combined signal is strong enough! the neuron 3fires!4 producing an output signal along the axon! which connects to dendrites of thousands of other neurons. 1ach input signal coming along a dendrite passes through a synapse or synaptic $unction! as shown. This $unction is an infinitesimal gap in the dendrite which is filled with neurotransmitter fluid that either accelerates or retards the flow of electrical charges. These electrical signals flow through the nucleus. The ad$ustment of the impedance or conductance of the synaptic gap by the neurotransmitter fluid contributes to the 3memory4 or 3learning process4 of the brain.

!"! Fro$ #u$an Neurons to Artificial Neurons

An artificial neuron is a concept whose components have direct analogy with the biological neuron. .igure /./shows the structure of an artificial neuron.

.igure /./ The neuron model &t is also called a neuron! P1 (processing element ! node or cell. The input signals 50! 5/... 5n are normally continuous variables but can also be discrete pulses. 1ach input signal flows through a gain or a weight! called a synaptic weight or connection strength! whose function is analogous to that of synaptic $unction in a biological neuron. The weights can be positive or negative corresponding to 3acceleration4 or 3inhibition4 respectively! of the flow of electrical signals in a biological cell. The summing node accumulates all the input weighted signals! then passes to the output through the activation function! which is usually non linear in nature! as shown in figure. 6athematically! the output expression can be given as!

," An en'ineerin' a--roac( ,") A si$-le neuron

The neuron has two modes of operation7 the training mode and the using mode. &n the training mode! the neuron can be trained to fire (or not ! for particular input patterns. &n the using mode! when a taught input pattern is detected at the input! its associated output becomes

the current output. &f the input pattern does not belong in the taught list of input patterns! the firing rule is used to determine whether to fire or not.

,"! Firin' rules

A firing rule determines how one calculates whether a neuron should fire for any input pattern. &t relates to all the input patterns! not only the ones on which the node was trained. A simple firing rule goes as follows8 Take a collection of training patterns for a node! some of which cause it to fire (the 0(taught set of patterns and others which prevent it from doing so (the 9(taught set . Then the patterns not in the collection cause the node to fire if! on comparison! they have more input elements in common with the -nearest- pattern in the 0( taught set than with the -nearest- pattern in the 9(taught set. &f there is a tie! then the pattern remains in the undefined state. .or example! a :(input neuron is taught to output 0 when the input (50! 5/ and 5: is 000 or 090 and to output 9 when the input is 999 or 990. Then! before applying the firing rule! the truth table is7 508 9 9 9 9 0 0 0 0 5/8 5:8 9 9 9 0 0 9 0 0 9 9 9 0 0 9 0 0

",T8 9 9 9;0 9;0 9;0 0 9;0 0 As an example of the way the firing rule is applied! take the pattern 909. &t differs from 999 in 0 element! from 990 in / elements! from 090 in : elements and from 000 in / elements. Therefore! the -nearest- pattern is 999 which belongs in the 9(taught set. Thus the firing rule re<uires that the neuron should not fire when the input is 990. "n the other hand! 900 is e<ually distant from two taught patterns that have different outputs and thus the output stays undefined (9;0 . 'y applying the firing in every column the following truth table is obtained7 508 9 9 9 9 0 0 0 0 5/8 5:8 ",T8 9 9 9 9 0 9 0 9 9 0 0 9;0 9 9 9;0 9 0 0 0 9 0 0 0 0

A--lications
The utility of artificial neural network models lies in the fact that they can be used to infer a function from observations. This is particularly useful in applications where the complexity of the data or task makes the design of such a function by hand impractical.

Real life a--lications

The tasks to which artificial neural networks are applied tend to fall within the following broad categories8 .unction approximation! or regression analysis! including time series prediction and modeling. +lassification! including pattern and se<uence recognition! novelty detection and se<uential decision making. +redit evaluation! marketing! business!electronic nose. Artificial neural networks deal with complex! often incomplete data! system identification and control (vehicle control! process control ! visual pattern recognition (radar systems! face identification! ob$ect recognition and more ! se<uence recognition (gesture! speech! handwritten text recognition ! medical diagnose malfunctions etc.

,", .attern Reco'nition % an e/a$-le

An important application of neural networks is pattern recognition. Pattern recognition can be implemented by using a feed(forward (figure :.: neural network that has been trained accordingly. =uring training! the network is trained to associate outputs with input patterns. >hen the network is used! it identifies the input pattern and tries to output the associated output pattern. The power of neural networks comes to life when a pattern that has no output associated with it! is given as an input. &n this case! the network gives the output that corresponds to a taught input pattern that is least different from the given pattern.

.igure :.: .or example8 The network of figure :.: is trained to recogni)e the patterns T and ?. The associated patterns are all black and all white respectively as shown below.

&f we represent black s<uares with 9 and white s<uares with 0 then the truth tables for the : neurons after generali)ation are7 5008 9 9 9 9 0 0 0 0 50/8 50:8 ",T8 5/08 5//8 5/:8 ",T8 9 9 9 9 9 9 0 9 0 9 9 9 0 9;0 0 9 0 0 9 9 9 0 9 9 9 0 9 0 9 0 9 0 9 0 0 0 9 9;0 0 0 0 0 0 0 9

0 0 To- neuron 9 9 0 9 0 0

0 9;0 9;0 0iddle neuron

*otto$ neuron

5/08 5//8 5/:8

9 9 9

9 9 0

9 0 9

9 0 0

0 9 9

0 9 0

0 0 9

0 0 0 9

",T8 0 9 0 0 9 9 0 .rom the tables it can be seen the following associations can be extracted8

&n this case! it is obvious that the output should be all blacks since the input pattern is almost the same as the -T- pattern.

?ere also! it is obvious that the output should be all whites since the input pattern is almost the same as the -?- pattern.

?ere! the top row is / errors away from the T and : from an ?. @o the top output is black. The middle row is 0 error away from both T and ? so the output is random. The bottom row is 0 error away from T and / away from ?. Therefore the output is black. The total output of the network is still in favour of the T shape.

,"1 A $ore co$-licated neuron"

The difference from the previous model is that the inputs are Aweighted27 the effect that each input has at decision making is dependent on the weight of the particular input. The weight of an input is a number which when multiplied with the input gives the weighted input. These weighted inputs are then added together and if they exceed a pre(set threshold value! the neuron fires. &n any other case the neuron does not fire.

.igure :.B An 6+P neuron &n mathematical terms! the neuron fires if and only if7 50>0 C 5/>/ C 5:>: C ... D T The addition of input weights and of the threshold makes this neuron a very flexible and powerful one.

1 Arc(itecture of neural networ s

1") Feed%forward networ s
.eed(forward ANNs (figure B.0 allow signals to travel one way only7 from input to output. There is no feedback (loops i.e. the output of any layer does not affect that same layer. .eed(forward ANNs tend to be straight forward networks that associate inputs with outputs. They are extensively used in pattern recognition. 1"! Feedbac networ s .eedback networks (figureB./ can have signals traveling in both directions by introducing loops in the network. .eedback networks are very powerful and can get extremely complicated. .eedback networks are dynamic7 their -state- is changing continuously until they reach an e<uilibrium point. They remain at the e<uilibrium point until the input changes and a new e<uilibrium needs to be found.

Fi'ure 1") An e/a$-le of a si$-le feed forward networ

Fi'ure 1"! An e/a$-le of a co$-licated networ

1", Networ la2ers

The commonest type of artificial neural network consists of three layers! of units8 a layer of 3in-ut4 units is connected to a layer of 3(idden4 units! which is connected to a layer of 3out-ut% units. (.igure B.0 The activity of the input units represents the raw information that is fed into the network. The activity of each hidden unit is determined by the activities of the input units and the weights on the connections between the input and the hidden units. The behaviour of the output units depends on the activity of the hidden units and the weights between the hidden and output units.

4" Conclusion
The computing world has a lot to gain from neural networks. Their ability to learn by example makes them very flexible and powerful. .urthermore there is no need to devise an algorithm in order to perform a specific task7 i.e. there is no need to understand the internal mechanisms of that task. They are also very well suited for real time systems because of their fast response and computational times which are due to their parallel architecture. Neural networks also contribute to other areas of research such as neurology and psychology. They are regularly used to model parts of living organisms and to investigate the internal mechanisms of the brain. Perhaps the most exciting aspect of neural networks is the possibility that some day -conscious- networks might be produced. There are a number of scientists arguing that consciousness is a -mechanical- property and that -conscious- neural networks are a realistic possibility.

#eferences8
0. 6odern Power 1lectronics and A+ =rives! 'imal E 'ose. /. Neural Networks! +hristos @tergiou and =imitrios @iganos. :. Neural Networks! >ikipedia! the free encyclopedia.htm"