SOFTCOMPUTINGTECHNIQUES      11

CHAPTER-3

SOFT COMPUTING TECHNIQUES

Soft Computing is the fusion of methodologies that were designed to model

and enable solutions to real world problems, which are not modeled or too difficult to
model, mathematically. Soft computing is a consortium of methodologies that works
synergistically and provides, in one form or another, flexible information processing
capability for handling real-life ambiguous situations [45]. Its aim is to exploit the
tolerance for imprecision, uncertainty, approximate reasoning and partial truth in
order to achieve tractability, robustness and low-cost solutions [35]. The guiding
principle is to devise methods of computation that lead to an acceptable solution at
low cost, by seeking for an approximate solution to an imprecisely or precisely
formulated problem [46].

Soft computing differs from conventional (hard) computing. Unlike hard

computing, it is tolerant of imprecision, uncertainty, partial truth and approximation.
In effect, the role model for soft computing is the human mind. Soft Computing is
basically optimization technique to find solution of problems which are very hard to
answer.

3.1 OPTIMIZATION

It is the process of making something better. Optimization is the process of

adjusting the inputs to find the minimum or maximum output or result [40]. The
Optimization Process is shown in figure 3.1.

Inputor Function Output

Variables or orCost
Process

Figure 3.1 Optimization Process.

Optimization is the mechanism by which one finds the maximum or minimum

value of a function or process. This mechanism is used in fields such as physics,
chemistry, economics, and engineering where the goal is to maximize efficiency,
SOFTCOMPUTINGTECHNIQUES      12

production or some other measure. Optimization can refer to either minimization or

maximization; maximization of a function f is equivalent to minimization of the
opposite of this function f.

An engineer or scientist conjures up a new idea and optimization improves on

that idea. Optimization consists in trying variations on an initial concept and using the
information gained to improve on the idea. A computer is the perfect tool for
optimization as long as the idea or variable influencing the idea can be input in
electronic format. Feed the computer some data and out comes the solution.

3.2 COMBINATORIAL OPTIMIZATION TECHNIQUES

Testing problem belongs to the some type of combinatorial optimization

problems. The techniques used to tackle combinatorial optimization problems can be
classified in two general category, firstly, the exact methods and secondly the
approximate (heuristic) methods [41]. Although exact methods granted solution to the
problem in hand but not appropriate for real life problems as it requires large
computation time seek because of their complex nature, hence the resolution by exact
methods is not realistic for large problems, justifying the use of powerful heuristic and
meta-heuristics methods. For practical use heuristic methods seek to find high quality
solutions (not necessarily optimal) within reasonable computation times [41]. Another
type of methods is meta-heuristics which have been applied successfully on large and
real life complex problems over the years by different researchers has provided
fruitful results [42 - 43]. Classification of common search methodologies are shown
in figures 3.2 and 3.3.

3.3 NEURAL NETWORKS (NNS)

There are millions of very simple processing elements or neurons in the brain,
linked together in a massively parallel manner. This is believed to be responsible for
the human intelligence and discriminating power [35]. Neural Networks are
developed to try to achieve biological system type performance using a dense
interconnection of simple processing elements analogous to biological neurons.
Neural Networks are information driven rather than data driven [47]. Typically, there
are at least two layers, an input layer and an output layer. One of the most common
networks is the Back Propagation Network (BPN) which consists of an input layer,
and an output layer with one or more intermediate hidden layers [41].
 SOFTCOMPUTINGTECHNIQUES      13

Neural Networks are trained to perform a particular function by adjusting the

values of the connections (weights) between elements using a set of examples before
they can be employed to the actual problem. Commonly neural networks are adjusted,
or trained, so that a particular input leads to a specific target output [44] which is
shown in figure 3.4. The method used to generate the examples to train the network
and the training algorithm employed has a significant impact on the performance of
the neural network-based model. One of the training algorithms used is the Back-
Propagation (BP) algorithm. This algorithm aims to reduce the deviation between the
desired objective function value and the actual objective function value [41].

Optimization

Continuous Combinatorial

Linear
Quadratic NonLinear Approximate
Method ExactMethod

LocalMethod GlobalMethod

Heuristic
ChemicalMethod
Meta
Heuristic

PopulationBased
NeighborhoodBased

Figure 3.2 Classifications of Common Search Methodologies

 SOFTCOMPUTINGTECHNIQUES      14

Metaheuristics

Population
Neighborhood BasedAlgorithm
BasedAlgorithm

TabuSearch Swarm
Simulated
Intelligence
Annealing

AntColony ParticleSwarm
Optimization Optimization

Evolutionary
computation

Evolutionary Genetic Genetic Evolutionary Differential

programming Algorithm Programming Strategies Algorithm

Figure 3.3 Classifications of Common Meta-heuristics.

The performance of this very slow and mostly trapped in local optima, there
are another training algorithms which has faster coverage speed and tries to avoid to
struck in local optima like Delta-bar-delta (DBD).
SOFTCOMPUTINGTECHNIQUES      15

Target

NNincluding 
Input connections(Weights) Output
Compare
betweenneurons

AdjustWeights

Figure 3.4 Neural Networks

Limitations The major issues of concern today are the scalability problem, testing,
verification and integration of neural network systems into the modern environment.
Neural network programs sometimes become unstable when applied to larger
problems. The defense, nuclear and space industries are concerned about the issue of
testing and verification. The mathematical theories used to guarantee the performance
of an applied neural network needs development. The solution for the time being may
be to train and test these intelligent systems much as we do for humans. Also there are
some more practical problems like: the operational problem encountered when
attempting to simulate the parallelism of neural networks. Since the majority of neural
networks are simulated on sequential machines, giving rise to a very rapid increase in
processing time requirements as size of the problem expands. Networks function as
"Black Boxes" whose rules of operation are completely unknown.

3.4 GENETIC ALGORITHMS (GAs)

The basic concepts were developed by Holland [8], while the practicality of
using the GA to solve complex problems was demonstrated in [49, 51]. Genetic
Algorithms (GAs) is a soft computing approach. GAs are general-purpose search
algorithms, which use principles inspired by natural genetics to evolve solutions to
problems [48]. As one can guess, genetic algorithms are inspired by Darwin's theory
about evolution. They have been successfully applied to a large number of scientific
and engineering problems, such as optimization, machine learning, automatic
programming, transportation problems, adaptive control etc.
SOFTCOMPUTINGTECHNIQUES      16

GA starts off with population of randomly generated chromosomes, each

representing a candidate solution to the concrete problem being solved and advances
towards better chromosomes by applying genetic operators based on the genetic
processes occurring in nature. So far, GAs had a great measure of success in search
and optimization problems due to their robust ability to exploit the information
accumulated about an initially unknown search space. Particularly GAs specialize in
large, complex and poorly understood search spaces where classic tools are
inappropriate, inefficient or time consuming.

As mentioned, the GA's basic idea is to maintain a population of

chromosomes. This population evolves over time through a successive iteration
process of competition and controlled variation. Each state of population is called
generation. Associated with each chromosome at every generation is a fitness value,
which indicates the quality of the solution, represented by the chromosome values.
Based upon these fitness values, the selection of the chromosomes, which form the
new generation, takes place. Like in nature, the new chromosomes are created using
genetic operators such as crossover and mutation.

Mechanism of GA

The fundamental mechanism consists of the following stages which is also

described in figure 3.5.

1. Generate randomly the initial population.

2. Select the chromosomes with the best fitness values.
3. Recombine selected chromosomes using crossover and mutation operators.
4. Insert offspring into the population.
5. If a stop criterion is satisfied, return the chromosome(s) with the best fitness.
Otherwise, go to Step 2.

In GA, the population is defined to be the collection of individuals. A

population is a generation that undergoes under changes to produce new generation.
Like nature, GAs have also collection of several members to make population healthy.
A chromosome that is a collection of genes is correspondence to individual of
population. Each individual chromosome represents a possible solution to the
optimization problem. The dimension of the GA refers to the dimension of the search
space which equals the number of genes in each chromosome.
SOFTCOMPUTINGTECHNIQUES      17

Initialize Selection
SelectIndividualsforMating
Population

Crossover
MateIndividuals

Mutation MutateOffspring

InsertInto InsertOffspringintoPopulation
Population

No Yes
Criteria Answer
Satisfied?

Figure 3.5 Fundamental Mechanism of Simple Genetic Algorithm

i) Representation of Chromosomes

The representation of chromosomes in GAs has very deep impact on the

performance of GA-based function. There are different methods of representation of
chromosomes like binary encoding, value encoding, permutation encoding, tree
encoding etc. The most commonly used encoding is binary encoding proposed by
Holland [8]. In this method, the value of individual is encoded as bit string consists of
binary values either 0 or 1. Each chromosome of population consists of same length
of binary string [36]. Suppose a program has two inputs X and Y having value 8 and
6 respectively and length of binary string is 5. Then, X and Y can be represented as
shown in table 3.1.
SOFTCOMPUTINGTECHNIQUES      18

Table 3.1 Binary Encoding

Chromosome Value Binary Encoding

X 8 01000
Y 6 00110

ii) Fitness

The fitness function is the bone marrow of every optimization problem. If

fitness function is not good, then no optimization methods help in finding the
solution. Therefore selection of fitness function is crucial issue. The fitness function
provides a way for the GA to analyze the performance of each chromosome in the
population. The GA assesses solutions for their quality according to the information
produced by this unit and not by using direct information about their structure. Given
a particular chromosome, the fitness function returns a single value, which represents
the merit of the corresponding solution to the problem. Fitness evaluation functions
might be complex or simple depending on the optimization problem at hand.

iii) Selection

The selection operator selects chromosomes from the current generation to be

parents for the next generation. The problem is how to select the chromosomes/
individuals. According to Darwins theory of evolution the best ones survive to create
new offspring.

Parents are selected in pairs. There are various types of selection methods which
are use to select the chromosomes:

Uniform Selection

In this selection method, parents are selected at random from a uniform

distribution using the expectations and number of parents. This results in an
undirected search. Uniform selection is not a useful search strategy, but it can be used
to test the genetic algorithm.

x Roulette Wheel Selection

This is a stochastic method, where individual having higher fitness value have
more chances to select. This is the most common selection method used in GAs.
Roulette wheel selection is shown in figure 3.6.
SOFTCOMPUTINGTECHNIQUES      19

Chromosome1
Chromosome2
Chromosome3
Chromosome4
Chromosome5

Figure 3.6 Roulette Wheel Selections

The algorithms of roulette wheel selection is given as

a. Do summation of fitness of all chromosomes and store this vale into S
b. Generate random number R between 0 and S.
c. Go through the population and sum the fitness from 0 - S. When the S is greater
than R, stop and return the chromosome where you are.
d. Repeat the process from b to c until you get number of chromosomes equal to the
size of population [36].
Tournament Selection

Tournament selection provides selection pressure by holding a tournament

among s competitors, where s is tournament size. The winner of the tournament is the
individual with highest fitness of the s tournament competitors. Then winner is
inserted into mating pool. The mating pool comprise of tournament winner, has a
higher fitness value than average population fitness [50].

The function selects each parent by choosing individuals at random, the

number of which you can specify by tournament size, and then choosing the best
individual out of that set to be a parent.

iv) Crossover

After reproduction simple crossover may proceed in two steps. First, members
of the newly reproduced strings in the mating pool are mated at random. Second each
SOFTCOMPUTINGTECHNIQUES      20

pair of chromosomes undergoes crossing over. Basically there are different types of
crossing over which is explained as:

Single Point Crossover

Crossover position is randomly selected between one and (L-1), where L is the
length of chromosome and two parents are crossed at that point. In this crossover, first
child is identical to first parent up to the crossing point and identical to the second
parent after the crossover point [62]. Figure 3.7 shows single point crossover.

Figure 3.7 Single Point Crossover

Two Point Crossover

In two points crossover, two crossover positions are randomly selected

between one and (L-1), where L is the length of chromosome and two parents are
crossed at those points. Two point crossover is shown in figure 3.8.

Figure 3.8 Two Point Crossovers

SOFTCOMPUTINGTECHNIQUES      21

x Uniform Crossover

Uniform crossover is shown in figure 3.9. In uniform crossover, bits are

randomly copied from the first or second parents [36].

1100 1011 0011 1110 1000 0000

+ =

0110 1000 1100 0100 1011 1111

Figure 3.9 Uniform Crossover

The amount of crossover is controlled by crossover probability Pc which is

defined as the ratio of the number of offspring produced in each generation to the
population size. A higher crossover probability allows exploration of more of the
solution space and reduces the chances of settling for a false optimum. A lower
crossover probability enables exploitation of existing individuals in the population
that have relatively high fitness [62].

v) Mutation

Mutations are global searches. A probability of mutation is again

predetermined before the algorithm is started which is applied to each individual bit
of each offspring chromosome to determine if it is to be inverted. As new individuals
are generated each character mutated with given probability Pm. In simple GA,
mutation means a 1 to a 0 and vice versa with mutation probability [48]. Figure 3.10
shows stages of chromosome before and after mutation.

Before Mutation 010100010010

After Mutation 011100000010

Figure 3.10 Mutation Operator

The mutation probability Pm controls the rate at which new gene values are
introduced into the population. If it is too small, many gene values that would have
been useful are never tried out. If it is too high, too much random perturbation will
occur and the offspring will lose their resemblance to the parents [62].

The mutation operator plays a secondary role in simple GA. Mutation rates are
smaller in natural populations, leading us to conclude that mutation is appropriately
considered as a secondary mechanism of genetic algorithm adaptation.
SOFTCOMPUTINGTECHNIQUES      22

3.5 PARTICLE SWARM OPTIMIZATION (PSO)

Although GAs provides good solution but they not keep information about the
best solution in the whole community. This strategy extends search by the
introduction of memory. In this optimization, along with the local best solution, a
global best solution is also stored somewhere in the memory, so that all particles not
trapped into local optima but moves to global optima.

PSO is an algorithm developed by Kennedy and Eberhart [16] that simulates

the social behaviors of bird flocking or fish schooling and the methods by which they
find roosting places, foods sources or other suitable habitat. The algorithm maintains a
population potential where each particle represents a potential solution to an
optimization problem. The PSO algorithm works by simultaneously maintaining
several candidate solutions in the search space. During each iteration of the algorithm,
each candidate solution is evaluated by the objective function being optimized,
determining the fitness of that solution. Each candidate solution can be thought of as a
particle flying through the fitness landscape finding the maximum or minimum of
the objective function. Initially, the PSO algorithm chooses candidate solutions
randomly within the search space. The initial state of a four-particle PSO algorithm
seeking the global maximum in a one-dimensional search space is shown in figure
3.11. The search space is composed of all the possible solutions along the x-axis. The
curve denotes the objective function. PSO algorithm has no knowledge of the
underlying objective function. Thus has no way of knowing if any of the candidate
solutions are near to or far away from a local or global maximum. PSO algorithm uses
the objective function to evaluate its candidate solutions and operates upon the
resultant fitness values.
Each particle maintains its position, composed of the candidate solution and its
evaluated fitness and its velocity. Additionally, it remembers the best fitness value it
has achieved thus far during the operation of the algorithm, referred to as the
individual best fitness and the candidate solution that achieved this fitness, referred to
as the individual best position or individual best candidate solution. Finally, PSO
algorithm maintains the best fitness value achieved among all particles in the swarm.
It is called the global best fitness. The candidate solution that achieved this fitness is
called the global best position or global best candidate solution.
SOFTCOMPUTINGTECHNIQUES      23

Figure 3.11 Initial PSO State

The PSO algorithm consists of just few steps, which are repeated until some
stopping condition is met. The steps are as follow:

1. Initialize the population, location and velocity.

2. Evaluate the fitness of the individual particle (Pbest).
3. Keep track of the individual highest fitness (Gbest).
4. Modify velocity based on Pbest and Gbest location.
5. Update the particle position.
6. Terminate if condition is met else go to step 3.

The first two steps are fairly trivial. Fitness evaluation is conducted by
supplying the candidate solution to the objective function. Individual, global best
fitnesses and positions are updated by comparing the newly evaluated fitnesses
against the previous individual and global best fitnesses, and replacing the best
fitnesses and positions as necessary. The velocity and position update step is
responsible for the optimization ability of the PSO algorithm.

The flowchart of PSO algorithm is shown in figure 3.12.

SOFTCOMPUTINGTECHNIQUES      24

Begin

Initializetheparticlespositionand
velocity

Calculatethefitnessofeveryparticle
N

Updatetheparticlespositionand
velocity
Criteriasatisfied

Generatereportandthenstop

Figure 3.12 Flow Chart of PSO

In the basic PSO technique, suppose that the search space is d-dimensional [21]

1. Each member is called particle, and each particle (i-th particle) is represented by d
dimensional vector and described as x i = [xi1, x i2,...,x id].
2. The set of n particle in the swarm are called population and described as pop=[x1,
x2,...,xd].
3. The best previous position for each particle (the position giving the best fitness
value) is called particle best and described as pbi =[pbi1, pbi2,...,pbid].
4. The best position among all of the particle best position achieved so far is called
global best and described as gbi =[gbi1, gbi2,...,gbid].
5. The rate of position change for each particle is called the particle velocity and
described as Vi =[Vi1, Vi2,...,Vid].

At iteration k the velocity for d-dimension of i-particle is updated by:

Vidk 1 wVidk  c1r1 ( pbidk  xidk )  c2 r2 ( gbidk  xidk ) (1)

SOFTCOMPUTINGTECHNIQUES      25

Where i= 1,2,..,n and n is the size of population, w is the inertia weight, c1 and c2
are the acceleration constants, and r1 and r2 are two random values in range [0,1].

6. The i-particle position is updated by:

x idk  1 x idk  V idk  1 (2)

Each of the three terms of the velocity update equation 1 has different roles in
the PSO algorithm. The first term wVKid is the inertia component, responsible for
keeping the particle moving in the same direction it was originally heading. The value
of the inertial coefficient w is typically between 0.8 and 1.2 which can either dampen
the particles inertia or accelerate the particle in its original direction. Generally,
lower values of the inertial coefficient speed up the convergence of the swarm to
optima, and higher values of the inertial coefficient encourage exploration of the
entire search space.

The second term c1r1(pbkid - xkid) called the cognitive component, acts as the
particles memory, causing it to tend to return to the regions of the search space in
which it has experienced high individual fitness. The cognitive coefficient c1 is
usually close to 2 and affects the size of the step the particle takes toward its
individual best candidate solution.

The third term c2r2(gbkid - xkid) called the social component, causes the particle
to move to the best region the swarm has found so far. The social coefficient c2 is
typically close to 2 and represents the size of the step the particle takes toward the
global best candidate solution gbkid the swarm has found up until that point.

The random values r1 in the cognitive component and r2 in the social

component cause these components to have a stochastic influence on the velocity
update. This stochastic nature causes each particle to move in a semi-random manner
heavily influenced in the directions of the individual best solution of the particle and
global best solution of the swarm.

PSO Algorithm to automatically generate test cases for the given program
defines:

Step 1: (Initialization):
Set the iteration number k=0. Generate randomly n particles, xi , i = 1, 2,..., n,
where x i = [xi1, xi2,...,x id]. and their initial velocities Vi =[Vi1, Vi2,...,Vid]. Evaluate the
SOFTCOMPUTINGTECHNIQUES      26

evaluation function for each particle eval (x i) using fitness function. If the constraints
are satisfied, then set the particle best PB i = x i and set the particle best which give the
best objective function among all the particle bests to global best gb. Else, repeat the
initialization.
Step 2: Update iteration counter k=k+1.
Step 3: Update velocity using Eq. (1).
Step 4: Update position using Eq. (2).
Step 5: Update particle best:
If eval i (xki ) > eval i (pbk1 i ) then
pbki = xki
Else pbki = pbk -1i
Step 6: Update global best:
eval(gbk) = max(eval i (pbk1 i ))
If eval(gbk) > eval(gbk1) then
gbk = gbk
Else gbk = gbk -1

Step 7: (Stopping criterion): If the number of iteration exceeds the maximum number
iteration or accumulated coverage is 100% then stop, otherwise go to step 2..

3.6 ANT COLONY OPTIMIZATION (ACO)

The idea of ant colony optimization is as its name suggests, inspired from the
ant colonies. Ant Colony Optimization (ACO) is a population-based, general search
technique for the solution of difficult combinatorial problems, which is inspired by
the pheromone trail laying behavior of real ant colonies [73]. Each ant moves along
some unknown path in search of food and while it goes it leaves behind a trail of what
is known as pheromone. The special feature of this pheromone is that it evaporates
with time such that as time proceeds, the concentration of the pheromone decreases on
any given path. Now its obvious that the path with maximum pheromone is the one
that has been traversed the most recently or in fact by most number of ants and hence
the most desirable for following ant [74]. The first ACO technique is known as Ant
System [75] and it was applied to the traveling salesman problem. This work was
further carried by Dorigo, Di Caro, Blum etc [75-79]. Initial attempts at an ACO
SOFTCOMPUTINGTECHNIQUES      27

algorithm were not very satisfying until the ACO algorithm was coupled with a local
optimizer. One problem is premature convergence to a less than optimal solution this
is because too much virtual pheromone was laid quickly. To avoid this stagnation,
pheromone evaporation is implemented. In other words, the pheromone associated
with a solution disappears after a period of time [40].

In ACO, a set of software agents called artificial ants search for good solutions
to a given optimization problem. The choice of a heuristic technique is quite justified,
as the use of any classic greedy approach shows very poor results [80]. The use of ant
colony optimization is best for the graph based problems [74].

To apply ACO, the optimization problem is transformed into the problem of

finding the best path on a weighted graph. The artificial ants (hereafter ants)
incrementally build solutions by moving on the graph. The solution construction
process is stochastic and is biased by a pheromone model, that is, a set of parameters
associated with graph components (either nodes or edges) whose values are modified
at runtime by the ants [73]. Algorithm for ACO is shown in figure 3.13.

Step 1: Initialization Initialize the pheromone trail

Step 2: Iteration
For each Ant Repeat
Solution construction using the current pheromone trail
Evaluate the solution constructed
Update the pheromone trail
Until stopping criteria

Figure 3.13 A Generic Ant Colony Algorithm

The ACO is a natural for the traveling salesperson problem. It begins with a
number of ants that follow a path around the different cities. Each ant deposits a
pheromone along the path. The algorithm begins by assigning each ant to a randomly
selected city. The next city is selected by a weighted probability that is a function of
the strength of the pheromone laid on the path and the distance of the city. The
probability that ant k will travel from city m to city n is given by






SOFTCOMPUTINGTECHNIQUES      28

Where

= pheromone strength

q = cities on tour k that come after city m

a = pheromone weighting; when a = 0, closest city is selected

b = distance weighting; when b = 0, distance between cities is ignored

Short paths with high pheromone have the highest probability of selection [40].