DHANALAKSHMI SRINIVASAN COLLEGE OF ENGINEERING

PERAMBALUR-621 212
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
EVEN SEMESTER (2015-2016)
REGULATION 2013
UNIT I
INTRODUCTION TO Al AND PRODUCTION SYSTEMS
Introduction to AI-Problem formulation, Problem Definition -Production systems, Control
strategies, Search strategies. Problem characteristics, Production system characteristics
Specialized production system- Problem solving methods - Problem graphs, Matching, Indexing
and Heuristic functions Hill Climbing-Depth first and Breath first, Constraints satisfaction Related algorithms, Measure of performance and analysis of search algorithms.
INTRODUCTION TO AI :
Many human mental activities such as developing computer programs, working
out mathematics, engaging in common sense reasoning, understanding languages and
interpreting it, even driving an automobile are said to demand Intelligence.
The central point of all such activities and systems is that How to think or rather
How to make system think. The process of thinking has various steps like perceive,
understand, predict and manipulate a world that is made up of tiny complex things or
situations.
The field of AI not just attempts to understand but also it builds intelligent
entities.
Artificial Intelligence (AI) is the study of how to make computers do things
which, at the moment, people do better.
PROBLEM FORMULATION:
Some game playing program used its past experience to improve its later
performance. Despite this, the computers could perform well at those tasks by finding
more number of solution paths and selecting the best one.
Some problem solving includes reasoning about physical objects and their
relationships to each other and reasoning about action and their consequences. For this,
logical expressions are symbolically manipulated.
1. MUNDANE TASKS:

Perception

Vision
Speech

Natural Language
Understanding
Generation
Transition
Commonsense Reasoning
Robot Control
2. FORMAL TASKS :
Games
Chess
backgammon
Checkers-GO
Mathematics
Geometry
Logic
Integral calculus
Proving properties of programs
3. EXPERT TASKS :
Engineering
Design
Fault Finding
Manufacturing planning
Science analysis
Medical Diagnosis
Financial Analysis
PROBLEM DEFINITION :
Four Necessity things to solve a problem:
Define the problem precisely
Analyze the problem
Isolate and represent the knowledge to solve the problem
Choose the best problem
Problem solving is fundamental to many AI-based applications.
There are two types of problems.

The Problems like, computation of the sine of an angle or the square root of
a value. These can be solved through the use of deterministic procedure
and the success is guaranteed.
In the real world,
very few problems
lend
themselves to straightforward solutions.

Problem solving is a process of generating solutions from observed data.

a problem is characterized by a set of goals,

a set of objects, and

a set of operations.

These could be ill-defined and may evolve during problem solving.

Problem space is an abstract space.

A problem space encompasses all valid states that can be generated by the
application of any combination of operators on any combination of objects.
The problem space may contain one or more solutions.

Search refers to the search for a solution in a problem space.

Search proceeds with different types of search control strategies.

The depth-first search and breadth-first search are the two common search
strategies.

GENERAL PROBLEM SOLVING

Problem solving has been the key areas of concern for Artificial Intelligence.
Problem solving is a process of generating solutions from observed or given
data. It is however not always possible to use direct methods (i.e. go directly from
data to solution). Instead, problem solving often need to use indirect or modelbased methods.
General Problem Solver (GPS) was a computer program created in 1957 by
Simon and Newell to build a universal problem solver machine. GPS was based on
Simon and Newell's theoretical work on logic machines. GPS in principle can
solve any formalized symbolic problem, like : theorems proof and geometric
problems and chess playing.
GPS solved many simple problems such as the Towers of Hanoi, that could be
sufficiently formalized, but GPS could not solve any real-world problems.
PROBLEM DEFINITIONS:
A problem is defined by its elements and their relations.
To provide a formal description of a problem, we need to do following:
a. Define a state space that contains all the possible configurations of the

relevant objects, including some impossible ones.

b. Specify one or more states, that describe possible situations, from which
the problem-solving process may start. These states are called initial states.
c. Specify one or more states that would be acceptable solution

to the problem. These states are called goal states.

d. Specify a set of rules that describe the actions (operators) available.

The problem can then be solved by using the rules, in combination with an
appropriate control strategy, to move through the problem space until a path from
an initial state to a goal state is found.
This process is known as search.

Search is fundamental to the problem-solving process.

Search is a general mechanism that can be used when more direct

method is not known.
Search provides the framework into which more direct methods for
solving subparts of a problem can be embedded.

A problem space is represented by directed graph, where nodes represent

search state and paths represent the operators applied to change the state.
To simplify a search algorithms, it is often convenient to logically and
programmatically represent a problem space as a tree. A tree usually
decreases the complexity of a search at a cost. Here, the cost is due to
duplicating some nodes on the tree that were linked numerous times in the
graph; e.g., node B and node D shown in example below.
A tree is a graph in which any two vertices are connected by exactly one
path. Alternatively, any connected graph with no cycles is a tree.

PROBLEM SOLVING:
The term Problem Solving relates analysis in AI. Problem solving may be
characterized as a systematic search through a range of possible actions to
reach some predefined goal or solution. Problem-solving methods
are
categorized as special purpose and general purpose.

Special-purpose method is tailor-made for a particular problem, often

exploits very specific features of the situation in which the problem is
embedded.
General-purpose method is applicable to a wide variety of problems. One
general-purpose technique used in AI is "means-end analysis". It is a step-bystep, or incremental, reduction of the difference between current state and final
goal.

Examples : Tower of Hanoi puzzle

For a Robot this might consist of PICKUP, PUTDOWN, MOVEFORWARD,

MOVEBACK, MOVELEFT, and MOVERIGHTuntil the goal is reached.

 Puzzles and Games have explicit rules : e.g., the Tower of Hanoi puzzle.

Final

START

This puzzle may involves a set of rings of different sizes that can
be placed on three different pegs.
The puzzle starts with the rings arranged as shown in Fig. (a)
The goal of this puzzle is to move them all as to Fig. (b)
Condition : Only the top ring on a peg can be moved, and it may only
be placed on a smaller ring, or on an empty peg.
In this Tower of Hanoi puzzle : Situations encountered while solving the
problem are described as states. The set of all possible configurations of rings
on the pegs is called problem space.
States
A state is a representation of elements at a given moment. A problem
is defined by its elements and their relations. At each instant of a problem, the
elements have specific descriptors and relations; the descriptors tell - how to
select elements ? Among all possible states, there are two special states called :

Initial state

- Final state
State Change:

is the start point

is the goal state
Successor Function

A Successor Function is needed for state change. The successor function moves one
state to another state.
Successor Function:
Is a description of possible actions; a set of operators.
Is a transformation function on a state representation, which converts that state into
another state.
Defines a relation of accessibility among states.
Represents the conditions of applicability of a state and corresponding

transformation function
Examples of Problem Definitions:
Example 1:
A game of

8Puzzle

1 2 3
4 5 6

State space:

configuration of 8-tiles on the board

7 8

Initial state: any configuration

Goal state: tiles in a specific order
Action: blank moves
ondition: the move is within the board
Transformation: blank movesLeft, Right, Up, Dn
Solution: optimal sequence of operators
Example 2:
A game of

n-queens puzzle; n= 8

State space: configurations n=8 queens on the board with only one queen per row
and column
Initial state: configuration without queens on the board
Goal state: configuration with n=8 queens such that no queen attacks any other
Operators or actions: place a queen on the board.
Condition: the new queen is not attacked by any other already placed
Transformation: place a new queen in a particular square of the board
Solution: one solution (cost is not considered)

PRODUCTION SYSTEMS:
A production system consists of:
A set of rules, each consisting of a left side that determines the applicability of the
rule and a right side that describes the operation to be performed if that rule is applied.
One or more knowledge/databases that contain whatever information is appropriate
for the particular task. Some parts of the database may be permanent, while other
parts of it may pertain only to the solution of the current problem.
A control strategy that specifies the order in which the rules will be compared to the
database and a way of resolving the conflicts that arise when several rules match at
once.
Major advantages of production systems for artificial intelligence

Separation of Knowledge and Control

A Natural Mapping onto State Space Search
Modularity of Production Rules
Pattern-Directed Control
Opportunities for Heuristic Control of Search
Tracing and Explanation
Language Independence
A Plausible Model of Human Problem-Solving

In order to solve a problem:

We must first reduce it to one for which a precise statement can be given. This can
be done by defining the problems state space ( start and goal states) and a set of
operators for moving that space.
The problem can then be solved by searching for a path through the space from an
initial state to a goal state.
The process of solving the problem can usefully be modelled as a production
system.
CONTROL STRATEGIES:
How to decide which rule to apply next during the process of searching for a solution to
a problem?

 The two requirements of good control strategy are that

it should cause motion.

It should be systematic

Breadth First Search

Algorithm:
1. Create a variable called NODE-LIST and set it to initial state
2. Until a goal state is found or NODE-LIST is empty do
a. Remove the first element from NODE-LIST and call it E. If NODE-LIST
was empty, quit
b. For each way that each rule can match the state described in E do:
i.

Apply the rule to generate a new state

ii.

If the new state is a goal state, quit and return this state

iii.

Otherwise, add the new state to the end of NODE-LIST

Algorithm: Depth First Search

1. If the initial state is a goal state, quit and return success
2. Otherwise, do the following until success or failure is signaled:
a. Generate a successor, E, of initial state. If there are no more successors, signal
failure.
b. Call Depth-First Search, with E as the initial state
c. If success is returned, signal success. Otherwise continue in this loop.
Backtracking
In this search, we pursue a signal branch of the tree until it yields a solution or until a
decision to terminate the path is made.
It makes sense to terminate a path if it reaches dead-end, produces a previous state. In
such a state backtracking occurs
Chronological Backtracking: Order in which steps are undone depends only on the
temporal sequence in which steps were initially made.
Specifically most recent step is always the first to be undone.
This is also simple backtracking.
Advantages of Depth-First Search

 DFS requires less memory since only the nodes on the current path are stored.
By chance, DFS may find a solution without examining much of the search space at all.
Advantages of BFS
BFS will not get trapped exploring a blind alley.
If there is a solution, BFS is guaranteed to find it.
If there are multiple solutions, then a minimal solution will be found.
TSP
A simple motion causing and systematic control structure could solve this problem.
Simply explore all possible paths in the tree and return the shortest path.
If there are N cities, then number of different paths among them is 1.2.(N-1) or (N-1)!
The time to examine single path is proportional to N
So the total time required to perform this search is proportional to N!
For 10 cities, 10! = 3,628,800
This phenomenon is called Combinatorial explosion.
Branch and Bound
Begin generating complete paths, keeping track of the shortest path found so far.
Give up exploring any path as soon as its partial length becomes greater than the shortest
path found so far.
Using this algorithm, we are guaranteed to find the shortest path.
It still requires exponential time.
The time it saves depends on the order in which paths are explored.

Heuristic Search
A Heuristic is a technique that improves the efficiency of a search process, possibly by
sacrificing claims of completeness.
Heuristics are like tour guides
They are good to the extent that they point in generally interesting directions;
They are bad to the extent that they may miss points of interest to particular individuals.
On the average they improve the quality of the paths that are explored.
Using Heuristics, we can hope to get good ( though possibly non optimal ) solutions to

hard problems such asa TSP in non exponential time.

There are good general purpose heuristics that are useful in a wide variety of problem
domains.
Special purpose heuristics exploit domain specific knowledge

Nearest Neighbor Heuristic

It works by selecting locally superior alternative at each step.
Applying to TSP:
1. Arbitrarily select a starting city
2. To select the next city, look at all cities not yet visited and select the one closest to
the current city. Go to next step.
3. Repeat step 2 until all cities have been visited.

This procedure executes in time proportional to N2

It
is possible to prove an upper bound on the error it incurs. This provides reassurance that
one is not paying too high a price in accuracy for speed.
PROBLEM CHARACTERISTICS
In order to choose the most appropriate method for a particular problem, it is necessary to
analyze the problem along several key dimensions:
Is the problem decomposable into a set of independent smaller or easier sub
problems?
Can solution steps be ignored or at least undone if they prove unwise?
Is the problems universe predictable?
Is a good solution to the problem obvious without comparison to all other possible
solutions?
Is the desired solution a state of the world or a path to a state?
Is a large amount of knowledge absolutely required to solve the problem or is
knowledge important only to constrain the search?
Can a computer that is simply given the problem return the solution or will the solution of the
problem require interaction between the computer and a person?
PRODUCTION SYSTEM CHARACTERISTICS
1. Can production systems, like problems, be described by a set of characteristics that shed
some light on how they can easily be implemented?
2. If so, what relationships are there between problem types and the types of production
systems best suited to solving the problems?
Classes of Production systems:

Monotonic Production System: the application of a rule never prevents the later
application of another rule that could also have been applied at the time the first
rule was selected.
Non-Monotonic Production system
Partially commutative Production system: property that if application of a
particular sequence of rules transforms state x to state y, then permutation of
those rules allowable, also transforms state x into state y.
Commutative Production system

SPECIALIZED PRODUCTION SYSTEM:

Monotonic Production Systems
Production system in which the application of a rule never prevents the later application
of another rule that could also have been applied at the time the first rule was applied.
i.e., rules are independent.
Commutative Production system
A partially Commutative production system has a property that if the application of a
particular sequence of rules transform state x into state y, then any permutation of those
rules that is allowable, also transforms state x into state y.
A Commutative production system is a production system that is both monotonic and
partially commutative.
Partially Commutative, Monotonic
These production systems are useful for solving ignorable problems.
Example: Theorem Proving
They can be implemented without the ability to backtrack to previous states when it is
discovered that an incorrect path has been followed.
This often results in a considerable increase in efficiency, particularly because since the
database will never have to be restored, It is not necessary to keep track of where in the
search process every change was made.
They are good for problems where things do not change; new things get created.
Non Monotonic, Partially Commutative
Useful for problems in which changes occur but can be reversed and in which order of
operations is not critical.
Example: Robot Navigation, 8-puzzle, blocks world
Suppose the robot has the following ops: go North (N), go East (E), go South (S), go
West (W). To reach its goal, it does not matter whether the robot executes the N-N-E or
N-E-N.
Not partially Commutative
Problems in which irreversible change occurs
Example: chemical synthesis
The ops can be :Add chemical x to the pot, Change the temperature to t degrees.
These ops may cause irreversible changes to the potion being brewed.
The order in which they are performed can be very important in determining the final
output.
(X+y) +z is not the same as (z+y) +x
Non partially commutative production systems are less likely to produce the same node
many times in search process.
When dealing with ones that describe irreversible processes, it is partially important to
make correct decisions the first time, although if the universe is predictable, planning can
be used to make that less important.

Advantages of production systems:1. Production systems provide an excellent tool for structuring AI programs.
2. Production Systems are highly modular because the individual rules can be added, removed or
modified independently.
3. The production rules are expressed in a natural form, so the statements contained in the
knowledge base should the a recording of an expert thinking out loud.
Disadvantages of Production Systems:One important disadvantage is the fact that it may be very difficult analyse the flow of control
within a production system because the individual rules dont call each other.
Production systems describe the operations that can be performed in a search for a solution to the
problem. They can be classified as follows.
Monotonic production system :- A system in which the application of a rule never prevents the
later application of another rule, that could have also been applied at the time the first rule was
selected.
SEARCH STRATEGIES
The strategies are evaluated based on 4 criteria:
1. Completeness: always find solution when there is one
2. Time Complexity: how long does it take to find a solution
3. Space Complexity: how much memory does it need to perform the search
4. Optimality: does the strategy find the highest-quality solution
PROBLEM SOLVING METHODS
Problem Reduction with AO* Algorithm.
PROBLEM REDUCTION ( AND - OR graphs - AO * Algorithm)
When a problem can be divided into a set of sub problems, where each sub problem can be
solved separately and a combination of these will be a solution, AND-OR graphs or AND - OR
trees are used for representing the solution. The decomposition of the problem or problem
reduction generates AND arcs. One AND are may point to any number of successor nodes. All
these must be solved so that the arc will rise to many arcs, indicating several possible solutions.
Hence the graph is known as AND - OR instead of AND. Figure shows an AND - OR graph.

AO* ALGORITHM:
1.

Let G consists only to the node representing the initial state call this node INTT. Compute
h' (INIT).

2. Until INIT is labeled SOLVED or hi (INIT) becomes greater than FUTILITY, repeat the
following procedure.
(I)

Trace the marked arcs from INIT and select an unbounded node NODE.

(II) Generate the successors of NODE . if there are no successors then assign FUTILITY as
h' (NODE). This means that NODE is not solvable. If there are successors then for each
one
called SUCCESSOR, that is not also an ancester of NODE do the following
(a) add SUCCESSOR to graph G
(b) if successor is not a terminal node, mark it solved and assign zero to its h ' value.
(c) If successor is not a terminal node, compute it h' value.
(III) propagate the newly discovered information up the graph by doing the following . let S be a
set of nodes that have been marked SOLVED. Initialize S to NODE. Until S is empty
repeat
the following procedure;
(a) select a node from S call if CURRENT and remove it from S.
(b) compute h' of each of the arcs emerging from CURRENT , Assign minimum h' to
CURRENT.
(c) Mark the minimum cost path a s the best out of CURRENT.
(d) Mark CURRENT SOLVED if all of the nodes connected to it through the new marked
are have been labeled SOLVED.
(e) If CURRENT has been marked SOLVED or its h ' has just changed, its new status
must
be propagate backwards up the graph . hence all the ancestors of CURRENT are added
to S.
AO* SEARCH PROCEDURE.
1. Place the start node on open.

2. Using the search tree, compute the most promising solution tree TP .
3. Select node n that is both on open and a part of tp, remove n from open and place it no closed.
4. If n is a goal node, label n as solved. If the start node is solved, exit with success where tp is
the solution tree, remove all nodes from open with a solved ancestor.
5. If n is not solvable node, label n as unsolvable. If the start node is labeled as unsolvable, exit
with failure. Remove all nodes from open ,with unsolvable ancestors.
6. Otherwise, expand node n generating all of its successor compute the cost of for each newly
generated node and place all such nodes on open.
7. Go back to step(2)
PROBLEM GRAPH:
AND/OR graphs
Some problems are best represented as achieving subgoals, some of which achieved
simultaneously and independently (AND)
Up to now, only dealt with OR options

Searching AND/OR graphs

A solution in an AND-OR tree is a sub tree whose leafs are included in the goal set
Cost function: sum of costs in AND node
f(n) = f(n1) + f(n2) + . + f(nk)
How can we extend A* to search AND/OR trees? The AO* algorithm.

MATCHING
To do so requires some kind of matching between the current state and the preconditions of the
rules.
How should this be done?
One way to select applicable rules is to do a simple search through all the rules comparing ones
preconditions to the current state and extracting all the ones that match . this requires indexing of
all the rules. But there are two problems with this simple solutions:
A. It requires the use of a large number of rules. Scanning through all of them would be
hopelessly in efficient.
B. It is not always immediately obvious whether a rules preconditions are satisfied by a
particular state.
Sometimes , instead of searching through the rules, we can use the current state as an index into
the rules and select the matching ones immediately. In spite of limitations, indexing in some
form is very important in the efficient operation of rules based systems.
A more complex matching is required when the preconditions of rule specify required properties
that are not stated explicitly in the description of the current state. In this case, a separate set of
rules must be used to describe how some properties can be inferred from others. An even more
complex matching process is required if rules should be applied and if their pre condition
approximately match the current situation. This is often the case in situations involving physical
descriptions of the world.
HEURISTIC FUNCTIONS:

A heuristic function is a function that maps from problem state description to measures
desirability, usually represented as number weights. The value of a heuristic function at a
given node in the search process gives a good estimate of that node being on the desired

path to solution.
Well designed heuristic functions can provides a fairly good estimate of whether a path is
good or not. ( " The sum of the distances traveled so far" is a simple heuristic function in
the traveling salesman problem) . the purpose of a heuristic function is to guide the search
process in the most profitable directions, by suggesting which path to follow first when
more than one path is available.
However in many problems, the cost of computing the value of a heuristic function
would be more than the effort saved in the search process. Hence generally there is a
trade-off between the cost of evaluating a heuristic function and the savings in search that
the function provides.

HILL CLIMBING :
Generate-and-Test
Algorithm
1. Generate a possible solution.
2. Test to see if this is actually a solution.
3. Quit if a solution has been found.
Otherwise, return to step 1.

Acceptable for simple problems.

Inefficient for problems with large space.
Exhaustive generate-and-test.
Heuristic generate-and-test: not consider paths that seem unlikely to lead to a solution.
Plan generate-test:
- Create a list of candidates.
- Apply generate-and-test to that list.

Example: coloured blocks

Arrange four 6-sided cubes in a row, with each side of each cube painted one of four
colours, such that on all four sides of the row one block face of each colour is showing.
Heuristic: if there are more red faces than other colours then, when placing a block with
several red faces, use few of them as possible as outside faces.
Hill Climbing
Searching for a goal state = Climbing to the top of a hill
Generate-and-test + direction to move.
Heuristic function to estimate how close a given state is to a goal state.
SIMPLE HILL CLIMBING
Algorithm
1. Evaluate the initial state.
2. Loop until a solution is found or there are no new operators left to be applied:

- Select and apply a new operator

- Evaluate the new state:
goal quit
better than current state new current state
Evaluation function as a way to inject task-specific knowledge into the control process.
STEEPEST-ASCENT HILL CLIMBING (GRADIENT SEARCH)
Considers all the moves from the current state.
Selects the best one as the next state.

Algorithm
1. Evaluate the initial state.
2. Loop until a solution is found or a complete iteration produces no change to current state:
- SUCC = a state such that any possible successor of the
current state will be better than SUCC (the worst state).
- For each operator that applies to the current state, evaluate
the new state:
goal quit
better than SUCC set SUCC to this state
- SUCC is better than the current state set the current
state to SUCC.
Hill Climbing: Disadvantages
Local maximum
A state that is better than all of its neighbours, but not better than some other states far away.
Plateau
A flat area of the search space in which all neighbouring states have the same value.
Ridge
The orientation of the high region, compared to the set of available moves, makes it
impossible to climb up. However, two moves executed serially may increase the height.
Ways Out
Backtrack to some earlier node and try going in a different direction.
Make a big jump to try to get in a new section.
Moving in several directions at once.
SIMULATED ANNEALING
A variation of hill climbing in which, at the beginning of the process, some downhill moves
may be made.
To do enough exploration of the whole space early on, so that the final solution is relatively
insensitive to the starting state.
Lowering the chances of getting caught at a local maximum, or plateau, or a ridge.
Physical Annealing
Physical substances are melted and then gradually cooled until some solid state is reached.

The goal is to produce a minimal-energy state.

Annealing schedule: if the temperature is lowered sufficiently slowly, then the goal will be
attained.
Nevertheless, there is some probability for a transition to a higher energy state: e-E/kT.
Algorithm
1. Evaluate the initial state.
2. Loop until a solution is found or there are no new operators left to be applied:
- Set T according to an annealing schedule
- Selects and applies a new operator
- Evaluate the new state:
goal quit
E = Val(current state) - Val(new state)
E < 0 new current state
Best-First Search
Depth-first search: not all competing branches having to be expanded.
Breadth-first search: not getting trapped on dead-end paths.
Combining the two is to follow a single path at a
time, but switch paths whenever
some competing
path look more promising than the current one.
OPEN: nodes that have been generated, but have not examined. This is organized as a
priority queue.
CLOSED: nodes that have already been examined. Whenever a new node is generated,
check whether it has been generated before.
Algorithm
1. OPEN = {initial state}.
2. Loop until a goal is found or there are no nodes left in OPEN:
- Pick the best node in OPEN
- Generate its successors
- For each successor:
new evaluate it, add it to OPEN, record its parent
generated before change parent, update successors
Greedy search:
h(n) = estimated cost of the cheapest path from node
n to a goal state.
Uniform-cost search:
g(n) = cost of the cheapest path from the initial state
Greedy search:
h(n) = estimated cost of the cheapest path from node
Neither optimal nor complete

to node n.
n to a goal state.

CONSTRAINT SATISFACTION
Many AI problems can be viewed as problems of constraint satisfaction.
Cryptarithmetic puzzle:

SEND
+ MORE
MONEY
As compared with a straight forward search procedure, viewing a problem as one of
constraint satisfaction can reduce substantially the amount of search.
Operates in a space of constraint sets.
Initial state contains the original constraints given in the problem.
A goal state is any state that has been constrained enough.
Two-step process:
1. Constraints are discovered and propagated as far as possible.
2. If there is still not a solution, then search begins, adding new constraints.
Initial state:
No two letters have the same value.
The sum of the digits must be as shown.
Two kinds of rules:
1. Rules that define valid constraint propagation.
2. Rules that suggest guesses when necessary.