A neural network, also known as a parallel distributed processing network, is a computing

solution that is loosely modeled after cortical structures of the brain. It consists of interconnected
processing elements called nodes or neurons that work together to produce an output function.
The output of a neural network relies on the cooperation of the individual neurons within the
network to operate. Processing of information by neural networks is characteristically done in
parallel rather than in series (or sequentially) as in earlier binary computers or Von Neumann
machines. Since it relies on its member neurons collectively to perform its function, a unique
property of a neural network is that it can still perform its overall function even if some of the
neurons are not functioning. In other words it is robust to tolerate error or failure. Additionally,
neural networks are more readily adaptable to fuzzy logic computing tasks than are Von
Neumann machines.1

Neural network theory is sometimes used to refer to a branch of computational science that uses
neural networks as models to either simulate or analyze complex phenomena and/or study the
principles of operation of neural networks analytically. It addresses problems similar to artificial
intelligence (AI) except that AI uses traditional computational algorithms to solve problems
whereas neural networks use 'networks of agents' (software or hardware entities linked together)
as the computational architecture to solve problems. Neural networks are trainable systems that
can "learn" to solve complex problems from a set of exemplars and generalize the "acquired
knowledge" to solve unforeseen problems as in stock market and environmental prediction. I.e.,
they are self-adaptive systems.2

History of the neural network analogy

The concept of neural networks started in the late-1800s as an effort to describe how the human
mind performed. These ideas started being applied to computational models with the Perceptron.

In early 1950s Friedrich Hayek was one of the first to posit the idea of spontaneous order in the
brain arising out of decentralized networks of simple units (neurons). In the late 1940s, Donald
Hebb made one of the first hypotheses for a mechanism of neural plasticity (i.e. learning),
Hebbian learning. Hebbian learning is considered be a 'typical' unsupervised learning rule and it
(and variants of it) was an early model for long term potentiation. The Perceptron is essentially a
linear classifier for classifying data specified by parameters

and an output function f = w'x + b. Its parameters are adapted with an ad-hoc rule similar to
stochastic steepest gradient descent. Because the inner product is linear operator in the input
space, the Perceptron can only perfectly classify a set of data for which different classes are
linearly separable in the input space, while it often fails completely for non-separable data. While
the development of the algorithm initially generated some enthusiasm, partly because of its
apparent relation to biological mechanisms, the later discovery of this inadequacy caused such
models to be abandoned until the introduction of non-linear models into the field.

The Cognitron was an early multilayered neural network with a training algorithm. The actual
structure of the network and the methods used to set the interconnection weights change from
one neural strategy to another, each with its advantages and disadvantages. Networks can
propagate information in one direction only, or they can bounce back and forth until self-
activation at a node occurs and the network settles on a final state. The ability for bi-directional
flow of inputs between neurons/nodes was produced with the Hopfield's network and
specialization of these node layers for specific purposes was introduced through the first hybrid
network.3

The parallel distributed processing of the mid-1980s became popular under the name
connectionism.4, 5

The back propagation network was probably the main reason behind the repopularisation of
neural networks after the publication of "Learning Internal Representations by Error
Propagation" in 1986. The original network utilised multiple layers of weight-sum units of the
type f = g(w'x + b), where g was a sigmoid function or logistic function such as used in logistic
regression. Training was done by a form of stochastic steepest gradient descent. The employment
of the chain rule of differentiation in deriving the appropriate parameter updates results in an
algorithm that seems to 'backpropagate errors', hence the nomenclature. However it is essentially
a form of gradient descent. Determining the optimal parameters in a model of this type is not
trivial, and steepest gradient descent methods cannot be relied upon to give the solution without a
good starting point. In recent times, networks with the same architecture as the backpropagation
network are referred to as Multi-Layer Perceptrons. This name does not impose any limitations
on the type of algorithm used for learning.

The backpropagation network generated much enthusiasm at the time and there was much
controversy about whether such learning could be implemented in the brain or not, partly
because a mechanism for reverse signalling was not obvious at the time, but most importantly
because there was no plausible source for the 'teaching' or 'target' signal.6,7, 8

Types of neural network

Traditionally, the term neural network has been used to refer to a network of biological neurons.
In modern usage, the term is often used to refer to artificial neural networks, which are composed
of artificial neurons or nodes. Thus the term 'Neural Network' has two distinct connotations:

1. Biological neural networks are made up of real biological neurons that are connected or
functionally-related in the peripheral nervous system or the central nervous system. In the field
of neuroscience, they are often identified as groups of neurons that perform a specific
physiological function in laboratory analysis.

2. Artificial neural networks are made up of interconnecting artificial neurons (usually simplified
neurons) designed to model (or mimic) some properties of biological neural networks. Artificial
neural networks can be used to model the modes of operation of biological neural networks,
whereas cognitive models are theoretical models that mimic cognitive brain functions without
necessarily using neural networks while artificial intelligence are well-crafted algorithms that
solve specific intelligent problems (such as chess playing, pattern recognition, etc.) without using
neural network as the computational architecture.9, 10
Fig: Simplified view of an artificial neural network

An (artificial) neural network consists of units, connections, and weights. Inputs and outputs are
numeric.

Biological NN Artificial NN
Soma Unit

axon, dendrite connection

synapse weight

potential weighted sum

threshold bias weight

signal activation

An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by

the way biological nervous systems, such as the brain, process information. The key element of
this paradigm is the novel structure of the information processing system. It is composed of a
large number of highly interconnected processing elements (neurones) working in unison to
solve specific problems. ANNs, like people, learn by example. An ANN is configured for a
specific application, such as pattern recognition or data classification, through a learning process.
Learning in biological systems involves adjustments to the synaptic connections that exist
between the neurones. This is true of ANNs as well.11, 12

Characterization

In general, a biological neural network is composed of a group or groups of physically connected

or functionally associated neurons. A single neuron can be connected to many other neurons and
the total number of neurons and connections in a network can be extremely large. Connections,
called synapses, are usually formed from axons to dendrites, though dendrodentritic
microcircuits and other connections are possible. Apart from the electrical signaling, there are
other forms of signaling that arise from neurotransmitter diffusion, which have an effect on
electrical signaling. As such, neural networks are extremely complex. While a detailed
description of neural systems seems currently unattainable, progress is made towards a better
understanding of basic mechanisms.
Artificial intelligence and cognitive modeling try to simulate some properties of neural networks.
While similar in their techniques, the former has the aim of solving particular tasks, while the
latter aims to build mathematical models of biological neural systems.

In the artificial intelligence field, artificial neural networks have been applied successfully to
speech recognition, image analysis and adaptive control, in order to construct software agents (in
computer and video games) or autonomous robots. Most of the currently employed artificial
neural networks for artificial intelligence are based on statistical estimation, optimisation and
control theory. The cognitive modelling field is the physical or mathematical modelling of the
behaviour of neural systems; ranging from the individual neural level (e.g. modelling the spike
response curves of neurons to a stimulus), through the neural cluster level (e.g. modelling the
release and effects of dopamine in the basal ganglia) to the complete organism (e.g. behavioural
modelling of the organism's response to stimuli).13

Learning process of Human Brain

In the human brain, a typical neuron collects signals from others through a host of fine structures
called dendrites. The neuron sends out spikes of electrical activity through a long, thin stand
known as an axon, which splits into thousands of branches. At the end of each branch, a structure
called a synapse converts the activity from the axon into electrical effects that inhibit or excite
activity from the axon into electrical effects that inhibit or excite activity in the connected
neurones. When a neuron receives excitatory input that is sufficiently large compared with its
inhibitory input, it sends a spike of electrical activity down its axon. Learning occurs by
changing the effectiveness of the synapses so that the influence of one neuron on another
changes.14

The brain, neural networks and computers

Historically computers evolved from Von Neumann architecture, based on sequential processing
and execution of explicit instructions. On the other hand origins of neural networks are based on
efforts to model information processing in biological systems, which are primarily based on
parallel processing as well as implicit instructions based on recognition of patterns of 'sensory'
input from external sources. In other words, rather than sequential processing and execution, at
their very heart, neural networks are complex statistic processors.15

Neural networks and Artificial intelligence

An artificial neural network (ANN), also called a simulated neural network (SNN) or commonly
just neural network (NN) is an interconnected group of artificial neurons that uses a
mathematical or computational model for information processing based on a connectionist
approach to computation. In most cases an ANN is an adaptive system that changes its structure
based on external or internal information that flows through the network. In more practical terms
neural networks are non-linear statistical data modeling tools. They can be used to model
complex relationships between inputs and outputs or to find patterns in data.16, 17

An artificial neural network involves a network of simple processing elements (neurons) which
can exhibit complex global behaviour, determined by the connections between the processing
elements and element parameters. One classical type of artificial neural network is the Hopfield
net.

In a neural network model, simple nodes (called variously "neurons", "neurodes", "PEs"
("processing elements") or "units") are connected together to form a network of nodes — hence
the term "neural network". While a neural network does not have to be adaptive per se, its
practical use comes with algorithms designed to alter the strength (weights) of the connections in
the network to produce a desired signal flow.

In modern software implementations of artificial neural networks the approach inspired by

biology has more or less been abandoned for a more practical approach based on statistics and
signal processing. In some of these systems neural networks, or parts of neural networks (such as
artificial neurons) are used as components in larger systems that combine both adaptive and non-
adaptive elements.18, 19

Neural networks and neuroscience

Theoretical and computational neuroscience is the field concerned with the theoretical analysis
and computational modeling of biological neural systems. Since neural systems are intimately
related to cognitive processes and behaviour, the field is closely related to cognitive and
behavioural modeling.

The aim of the field is to create models of biological neural systems in order to understand how
biological systems work. To gain this understanding, neuroscientists strive to make a link
between observed biological processes (data), biologically plausible mechanisms for neural
processing and learning (biological neural network models) and theory (statistical learning
theory and information theory).

Current research

While initially research had been concerned mostly with the electrical characteristics of neurons,
a particularly important part of the investigation in recent years has been the exploration of the
role of neuromodulators such as dopamine, acetylcholine, and serotonin on behaviour and
learning.20

Applications of neural networks

The utility of artificial neural network models lies in the fact that they can be used to infer a
function from observations. This is particularly useful in applications where the complexity of
the data or task makes the design of such a function by hand impractical.

Real life applications

The tasks to which artificial neural networks are applied tend to fall within the following broad
categories:

· Function approximation, or regression analysis, including time series prediction and

modelling.

· Classification, including pattern and sequence recognition, novelty detection and sequential
decision making.

· Data processing, including filtering, clustering, blind signal separation and compression. 21, 22

1 Neural Networks in Practice

Neural networks have broad applicability to real world business problems. In fact, they have
already been successfully applied in many industries.

Since neural networks are best at identifying patterns or trends in data, they are well suited for
prediction or forecasting needs including:

·sales forecasting

·industrial process control

·customer research

·data validation

·risk management

·target marketing

But to give you some more specific examples; ANN are also used in the following specific
paradigms: recognition of speakers in communications; diagnosis of hepatitis; recovery of
telecommunications from faulty software; interpretation of multimeaning Chinese words;
undersea mine detection; texture analysis; three-dimensional object recognition; hand-written
word recognition; and facial recognition.23, 24, 25

2 Neural networks in medicine

Artificial Neural Networks (ANN) is currently a 'hot' research area in medicine and it is believed
that they will receive extensive application to biomedical systems in the next few years. At the
moment, the research is mostly on modelling parts of the human body and recognising diseases
from various scans (e.g. cardiograms, CAT scans, ultrasonic scans, etc.).

Neural networks are ideal in recognising diseases using scans since there is no need to provide a
specific algorithm on how to identify the disease. Neural networks learn by example so the
details of how to recognize the disease are not needed. What is needed is a set of examples that
are representative of all the variations of the disease. The quantity of examples is not as
important as the 'quantity'. The examples need to be selected very carefully if the system is to
perform reliably and efficiently.26

2.1 Modelling and Diagnosing the Cardiovascular System

Neural Networks are used experimentally to model the human cardiovascular system. Diagnosis
can be achieved by building a model of the cardiovascular system of an individual and
comparing it with the real time physiological measurements taken from the patient. If this routine
is carried out regularly, potential harmful medical conditions can be detected at an early stage
and thus make the process of combating the disease much easier.

A model of an individual's cardiovascular system must mimic the relationship among

physiological variables (i.e., heart rate, systolic and diastolic blood pressures, and breathing rate)
at different physical activity levels. If a model is adapted to an individual, then it becomes a
model of the physical condition of that individual. The simulator will have to be able to adapt to
the features of any individual without the supervision of an expert. This calls for a neural
network.

Another reason that justifies the use of ANN technology is the ability of ANNs to provide sensor
fusion which is the combining of values from several different sensors. Sensor fusion enables the
ANNs to learn complex relationships among the individual sensor values, which would
otherwise be lost if the values were individually analysed. In medical modelling and diagnosis,
this implies that even though each sensor in a set may be sensitive only to a specific
physiological variable, ANNs are capable of detecting complex medical conditions by fusing the
data from the individual biomedical sensors.27,28

2.2 Electronic noses

ANNs are used experimentally to implement electronic noses. Electronic noses have several
potential applications in telemedicine. Telemedicine is the practice of medicine over long
distances via a communication link. The electronic nose would identify odours in the remote
surgical environment. These identified odours would then be electronically transmitted to
another site where a door generation system would recreate them. Because the sense of smell can
be an important sense to the surgeon, telesmell would enhance telepresent surgery.29,30

2.3 Instant Physician

An application developed in the mid-1980s called the "instant physician" trained an
autoassociative memory neural network to store a large number of medical records, each of
which includes information on symptoms, diagnosis, and treatment for a particular case. After
training, the net can be presented with input consisting of a set of symptoms; it will then find the
full stored pattern that represents the "best" diagnosis and treatment.(31)

3. Neural Networks in business

Business is a diverted field with several general areas of specialisation such as accounting or
financial analysis. Almost any neural network application would fit into one business area or
financial analysis.
There is some potential for using neural networks for business purposes, including resource
allocation and scheduling. There is also a strong potential for using neural networks for database
mining, that is, searching for patterns implicit within the explicitly stored information in
databases. Most of the funded work in this area is classified as proprietary. Thus, it is not
possible to report on the full extent of the work going on. Most work is applying neural
networks, such as the Hopfield-Tank network for optimization and scheduling.32

3.1 Marketing

There is a marketing application which has been integrated with a neural network system. The
Airline Marketing Tactician (a trademark abbreviated as AMT) is a computer system made of
various intelligent technologies including expert systems. A feedforward neural network is
integrated with the AMT and was trained using back-propagation to assist the marketing control
of airline seat allocations. The adaptive neural approach was amenable to rule expression.
Additionaly, the application's environment changed rapidly and constantly, which required a
continuously adaptive solution. The system is used to monitor and recommend booking advice
for each departure. Such information has a direct impact on the profitability of an airline and can
provide a technological advantage for users of the system. While it is significant that neural
networks have been applied to this problem, it is also important to see that this intelligent
technology can be integrated with expert systems and other approaches to make a functional
system. Neural networks were used to discover the influence of undefined interactions by the
various variables. While these interactions were not defined, they were used by the neural system
to develop useful conclusions. It is also noteworthy to see that neural networks can influence the
bottom line.33,34

3.2 Credit Evaluation

The HNC Company, founded by Robert Hecht-Nielsen, has developed several neural network
applications. One of them is the Credit Scoring system which increases the profitability of the
existing model up to 27%. The HNC neural systems were also applied to mortgage screening. A
neural network automated mortgage insurance underwritting system was developed by the
Nestor Company. This system was trained with 5048 applications of which 2597 were certified.
The data related to property and borrower qualifications. In a conservative mode the system
agreed on the underwritters on 97% of the cases. In the liberal model the system agreed 84% of
the cases. This is system run on an Apollo DN3000 and used 250K memory while processing a
case file in approximately 1 sec.35

Conclusion

The computing world has a lot to gain from neural networks. Their ability to learn by example
makes them very flexible and powerful. Furthermore there is no need to devise an algorithm in
order to perform a specific task; i.e. there is no need to understand the internal mechanisms of
that task. They are also very well suited for real time systems because of their fast response and
computational times which are due to their parallel architecture.

Neural networks also contribute to other areas of research such as neurology and psychology.
They are regularly used to model parts of living organisms and to investigate the internal
mechanisms of the brain.

Perhaps the most exciting aspect of neural networks is the possibility that some day 'conscious'
networks might be produced. There is a number of scientists arguing that consciousness is a
'mechanical' property and that 'consious' neural networks are a realistic possibility.

Even though neural networks have a huge potential we will only get the best of them when they
are integrated with computing, AI, fuzzy logic and related subjects.