1.

Artificial neural network

Artificial neural networks (ANN) or connectionist systems are computing systems that are inspired
by, but not necessarily identical to, the biological neural networks that constitute animal brains. Such
systems "learn" to perform tasks by considering examples, generally without being programmed with
any task-specific rules. For example, in image recognition, they might learn to identify images that
contain cats by analyzing example images that have been manually labeled as "cat" or "no cat" and
using the results to identify cats in other images. They do this without any prior knowledge about cats,
for example, that they have fur, tails, whiskers and cat-like faces. Instead, they automatically generate
identifying characteristics from the learning material that they process.
An ANN is based on a collection of connected units or nodes called artificial neurons, which loosely
model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can
transmit a signal from one artificial neuron to another. An artificial neuron that receives a signal can
process it and then signal additional artificial neurons connected to it.
In common ANN implementations, the signal at a connection between artificial neurons is a real
number, and the output of each artificial neuron is computed by some non-linear function of the sum
of its inputs. The connections between artificial neurons are called 'edges'. Artificial neurons and
edges typically have a weight that adjusts as learning proceeds. The weight increases or decreases the
strength of the signal at a connection. Artificial neurons may have a threshold such that the signal is
only sent if the aggregate signal crosses that threshold. Typically, artificial neurons are aggregated
into layers. Different layers may perform different kinds of transformations on their inputs. Signals
travel from the first layer (the input layer), to the last layer (the output layer), possibly after traversing
the layers multiple times.
The original goal of the ANN approach was to solve problems in the same way that a human brain
would. However, over time, attention moved to performing specific tasks, leading to deviations from
biology. Artificial neural networks have been used on a variety of tasks, including computer vision,
speech recognition, machine translation, social network filtering, playing board and video games and
medical diagnosis.

An artificial neural network is an interconnected group of nodes, inspired by a simplification of

neurons in a brain. Here, each circular node represents an artificial neuron and an arrow represents a
connection from the output of one artificial neuron to the input of another.
2. Fuzzy logic
Fuzzy logic is a form of many-valued logic in which the truth values of variables may be any real
number between 0 and 1 inclusive. It is employed to handle the concept of partial truth, where the
truth value may range between completely true and completely false.[1] By contrast, in Boolean logic,
the truth values of variables may only be the integer values 0 or 1.
The term fuzzy logic was introduced with the 1965 proposal of fuzzy set theory by Lotfi Zadeh.[2][3]
Fuzzy logic had however been studied since the 1920s, as infinite-valued logic—notably by
Łukasiewicz and Tarski.[4]
It is based on the observation that people make decisions based on imprecise and non-numerical
information, fuzzy models or sets are mathematical means of representing vagueness and imprecise
information, hence the term fuzzy. These models have the capability of recognising, representing,
manipulating, interpreting, and utilising data and information that are vague and lack certainty.[5]
Fuzzy logic has been applied to many fields, from control theory to artificial intelligence.
Fuzzification
Fuzzification is the process of assigning the numerical input of a system to fuzzy sets with some
degree of membership. This degree of membership may be anywhere within the interval [0,1]. If it is
0 then the value does not belong to the given fuzzy set, and if it is 1 then the value completely belongs
within the fuzzy set. Any value between 0 and 1 represents the degree of uncertainty that the value
belongs in the set. These fuzzy sets are typically described by words, and so by assigning the system
input to fuzzy sets, we can reason with it in a linguistically natural manner.
For example, in the image below the meanings of the expressions cold, warm, and hot are represented
by functions mapping a temperature scale. A point on that scale has three "truth values"—one for each
of the three functions. The vertical line in the image represents a particular temperature that the three
arrows (truth values) gauge. Since the red arrow points to zero, this temperature may be interpreted as
"not hot"; i.e. this temperature has zero membership in the fuzzy set "hot". The orange arrow
(pointing at 0.2) may describe it as "slightly warm" and the blue arrow (pointing at 0.8) "fairly cold".
Therefore, this temperature has 0.2 membership in the fuzzy set "warm" and 0.8 membership in the
fuzzy set "cold". The degree of membership assigned for each fuzzy set is the result of fuzzification.

Fuzzy logic temperature

Fuzzy sets are often defined as triangle or trapezoid-shaped curves, as each value will have a slope
where the value is increasing, a peak where the value is equal to 1 (which can have a length of 0 or
greater) and a slope where the value is decreasing.[citation needed] They can also be defined using a
sigmoid function.[9] One common case is the standard logistic function
Fuzzy logic operators
Fuzzy logic works with membership values in a way that mimics Boolean logic. To this end,
replacements for basic operators AND, OR, NOT must be available. There are several ways to this. A
common replacement is called the Zadeh operators:
Boolean Fuzzy
AND(x,y) MIN(x,y)
OR(x,y) MAX(x,y)
NOT(x) 1–x
For TRUE/1 and FALSE/0, the fuzzy expressions produce the same result as the Boolean expressions.
There are also other operators, more linguistic in nature, called hedges that can be applied. These are
generally adverbs such as very, or somewhat, which modify the meaning of a set using a mathematical
formula.[citation needed]
However, an arbitrary choice table does not always define a fuzzy logic function. In the paper, [10] a
criterion has been formulated to recognize whether a given choice table defines a fuzzy logic function
and a simple algorithm of fuzzy logic function synthesis has been proposed based on introduced
concepts of constituents of minimum and maximum. A fuzzy logic function represents a disjunction
of constituents of minimum, where a constituent of minimum is a conjunction of variables of the
current area greater than or equal to the function value in this area (to the right of the function value in
the inequality, including the function value).
Another set of AND/OR operators is based on multiplication
x AND y = x*y
x OR y = 1-(1-x)*(1-y) = x+y-x*y

comes from this:

x OR y = NOT( AND( NOT(x), NOT(y) ) )
x OR y = NOT( AND(1-x, 1-y) )
x OR y = NOT( (1-x)*(1-y) )
x OR y = 1-(1-x)*(1-y)
IF-THEN rules
Main article: Fuzzy rule
IF-THEN rules map input or computed truth values to desired output truth values. Example:
IF temperature IS very cold THEN fan_speed is stopped
IF temperature IS cold THEN fan_speed is slow
IF temperature IS warm THEN fan_speed is moderate
IF temperature IS hot THEN fan_speed is high
Given a certain temperature, the fuzzy variable hot has a certain truth value, which is copied to the
high variable.
Should an output variable occur in several THEN parts, then the values from the respective IF parts
are combined using the OR operator.
Defuzzification
Main article: Defuzzification
The goal is to get a continuous variable from fuzzy truth values.[citation needed]
This would be easy if the output truth values were exactly those obtained from fuzzification of a given
number. Since, however, all output truth values are computed independently, in most cases they do
not represent such a set of numbers.[citation needed] One has then to decide for a number that matches best
the "intention" encoded in the truth value. For example, for several truth values of fan_speed, an
actual speed must be found that best fits the computed truth values of the variables 'slow', 'moderate'
and so on.[citation needed]
There is no single algorithm for this purpose.
A common algorithm is
1. For each truth value, cut the membership function at this value
2. Combine the resulting curves using the OR operator
3. Find the center-of-weight of the area under the curve
4. The x position of this center is then the final output.
3. Genetic algorithm
In computer science and operations research, a genetic algorithm (GA) is a metaheuristic inspired by
the process of natural selection that belongs to the larger class of evolutionary algorithms (EA).
Genetic algorithms are commonly used to generate high-quality solutions to optimization and search
problems by relying on bio-inspired operators such as mutation, crossover and selection.[1] John
Holland introduced genetic algorithms in 1960 based on the concept of Darwin’s theory of evolution;
afterwards, his student David E. Goldberg extended GA in 1989.[2]
Optimization problems
In a genetic algorithm, a population of candidate solutions (called individuals, creatures, or
phenotypes) to an optimization problem is evolved toward better solutions. Each candidate solution
has a set of properties (its chromosomes or genotype) which can be mutated and altered; traditionally,
solutions are represented in binary as strings of 0s and 1s, but other encodings are also possible.[3]
The evolution usually starts from a population of randomly generated individuals, and is an iterative
process, with the population in each iteration called a generation. In each generation, the fitness of
every individual in the population is evaluated; the fitness is usually the value of the objective
function in the optimization problem being solved. The more fit individuals are stochastically selected
from the current population, and each individual's genome is modified (recombined and possibly
randomly mutated) to form a new generation. The new generation of candidate solutions is then used
in the next iteration of the algorithm. Commonly, the algorithm terminates when either a maximum
number of generations has been produced, or a satisfactory fitness level has been reached for the
population.
A typical genetic algorithm requires:
1. a genetic representation of the solution domain,
2. a fitness function to evaluate the solution domain.
A standard representation of each candidate solution is as an array of bits.[3] Arrays of other types and
structures can be used in essentially the same way. The main property that makes these genetic
representations convenient is that their parts are easily aligned due to their fixed size, which facilitates
simple crossover operations. Variable length representations may also be used, but crossover
implementation is more complex in this case. Tree-like representations are explored in genetic
programming and graph-form representations are explored in evolutionary programming; a mix of
both linear chromosomes and trees is explored in gene expression programming.
Once the genetic representation and the fitness function are defined, a GA proceeds to initialize a
population of solutions and then to improve it through repetitive application of the mutation,
crossover, inversion and selection operators.
Initialization
The population size depends on the nature of the problem, but typically contains several hundreds or
thousands of possible solutions. Often, the initial population is generated randomly, allowing the
entire range of possible solutions (the search space). Occasionally, the solutions may be "seeded" in
areas where optimal solutions are likely to be found.
Selection
Main article: Selection (genetic algorithm)
During each successive generation, a portion of the existing population is selected to breed a new
generation. Individual solutions are selected through a fitness-based process, where fitter solutions (as
measured by a fitness function) are typically more likely to be selected. Certain selection methods rate
the fitness of each solution and preferentially select the best solutions. Other methods rate only a
random sample of the population, as the former process may be very time-consuming.
The fitness function is defined over the genetic representation and measures the quality of the
represented solution. The fitness function is always problem dependent. For instance, in the knapsack
problem one wants to maximize the total value of objects that can be put in a knapsack of some fixed
capacity. A representation of a solution might be an array of bits, where each bit represents a different
object, and the value of the bit (0 or 1) represents whether or not the object is in the knapsack. Not
every such representation is valid, as the size of objects may exceed the capacity of the knapsack. The
fitness of the solution is the sum of values of all objects in the knapsack if the representation is valid,
or 0 otherwise.
In some problems, it is hard or even impossible to define the fitness expression; in these cases, a
simulation may be used to determine the fitness function value of a phenotype (e.g. computational
fluid dynamics is used to determine the air resistance of a vehicle whose shape is encoded as the
phenotype), or even interactive genetic algorithms are used.
Genetic operators
Main articles: Crossover (genetic algorithm) and Mutation (genetic algorithm)
The next step is to generate a second generation population of solutions from those selected through a
combination of genetic operators: crossover (also called recombination), and mutation.
For each new solution to be produced, a pair of "parent" solutions is selected for breeding from the
pool selected previously. By producing a "child" solution using the above methods of crossover and
mutation, a new solution is created which typically shares many of the characteristics of its "parents".
New parents are selected for each new child, and the process continues until a new population of
solutions of appropriate size is generated. Although reproduction methods that are based on the use of
two parents are more "biology inspired", some research[4][5] suggests that more than two "parents"
generate higher quality chromosomes.
These processes ultimately result in the next generation population of chromosomes that is different
from the initial generation. Generally the average fitness will have increased by this procedure for the
population, since only the best organisms from the first generation are selected for breeding, along
with a small proportion of less fit solutions. These less fit solutions ensure genetic diversity within the
genetic pool of the parents and therefore ensure the genetic diversity of the subsequent generation of
children.
Opinion is divided over the importance of crossover versus mutation. There are many references in
Fogel (2006) that support the importance of mutation-based search.
Although crossover and mutation are known as the main genetic operators, it is possible to use other
operators such as regrouping, colonization-extinction, or migration in genetic algorithms.[6]
It is worth tuning parameters such as the mutation probability, crossover probability and population
size to find reasonable settings for the problem class being worked on. A very small mutation rate
may lead to genetic drift (which is non-ergodic in nature). A recombination rate that is too high may
lead to premature convergence of the genetic algorithm. A mutation rate that is too high may lead to
loss of good solutions, unless elitist selection is employed.
Heuristics
In addition to the main operators above, other heuristics may be employed to make the calculation
faster or more robust. The speciation heuristic penalizes crossover between candidate solutions that
are too similar; this encourages population diversity and helps prevent premature convergence to a
less optimal solution.[7][8]
Termination
This generational process is repeated until a termination condition has been reached. Common
terminating conditions are:
 A solution is found that satisfies minimum criteria
 Fixed number of generations reached
 Allocated budget (computation time/money) reached
 The highest ranking solution's fitness is reaching or has reached a plateau such that successive
iterations no longer produce better results
 Manual inspection
 Combinations of the above