AlphaGo vs.

Lee Sedol 4-1

IT University of Copenhagen

Rune Mller Jensen

Lecture 7: Constraint Programming I

IT University of Copenhagen

Todays Program
[12:00-12:55]
Constraint Satisfaction Problems (CSPs)
Constraint Propagation

[13:05-14:00]
Backtracking
Forward Checking
Maintaining arc consistency (MAC) algorithm

[14:00-16:00]
Exercises

IT University of Copenhagen

Rune Mller Jensen

Constraint Satisfaction Problems (CSPs)

IT University of Copenhagen

Rune Mller Jensen

A CSP from Hell

IT University of Copenhagen

Rune Mller Jensen

Definition of CSPs
A CSP is a triple <X,D,C>, where:
X = {X1,,Xn} is a finite set of variables.
D = {D1,,Dn} is a set of domains of possible values for each
variable, where Di = {v1,,vki}
C = {C1,,Cm} is a set of constraints, where Ci = <scope, relation>
e.g.

X1{A,B}, X2{A,B}

Implicit constraint representation: <(X1,X2), X1 X2>

Explicit constraint representation: <(X1,X2), [(A,B), (B,A)]>
Typical short notation: X1 X2
IT University of Copenhagen

Rune Mller Jensen

CSP Solutions
Partial Assignment : Values assigned to only some
of the variables.
Complete Assignment : Each variable has a value
assigned.

Consistent Assignment : Each constraint, where all

variables in its scope are assigned, is satisfied.
(obs: not clearly stated in RN13!)
Solution : Complete consistent assignment.

IT University of Copenhagen

Rune Mller Jensen

Types of Constraints
Different Arity
Unary constraints involve a single variable.
e.g. X 12

Binary constraints involve a pair of variables.

e.g. X > Y,

P = MIB /\ C = Black

Global constraints involve an arbitrary number of

variables.
e.g. AllDifferent

Today we assume constraints to be unary or binary!

IT University of Copenhagen

Rune Mller Jensen

CSP Example: Map Coloring

Variables: WA, NT, Q, NSW, V, SA, T

Domains: Di = {red, green, blue}

Constraints: adjacent regions must have different colors.
e.g. <(WA,NT), WA NT>
<(WA,NT) , [(red, green), (red, blue), (green, red), ]>
IT University of Copenhagen

Rune Mller Jensen

CSP Example: Map Coloring

Solutions are assignments satisfying all constraints, e.g.

{WA=red, NT=green, Q=red, NSW=green, V=red, SA=blue,

T=green}
IT University of Copenhagen

Rune Mller Jensen

10

Constraint Graph Representation of CSP

Nodes are variables

Edges are binary constraints

IT University of Copenhagen

Rune Mller Jensen

11

Constraint Propagation (Rule Inference)

CSPs are solved combining search
and constraint propagation

IT University of Copenhagen

Rune Mller Jensen

12

Constraint Propagation
Node consistency : for every value vi of
variable X, all unary constraints of X are
satisfied.

Example
SA {red, green, blue}, SA {green}
SA {red, blue}

Arc consistency: for every X Y arc, every

value vi in X has a support value uj in Y.
IT University of Copenhagen

Rune Mller Jensen

13

Arc Consistency Examples

1) SA {red, green, blue}, NT {blue}, SA NT
SA NT arc-consistent: SA {red, green}
2) X,Y {0,1,...,9}, Y=X2
X Y arc-consistent : X {0,1,2,3} , Y {0,1,...,9}
Y X arc-consistent: X {0,1,,9} , Y {0,1,4,9}
3) SA {red, green, blue}, WA {red, green, blue}, SA WA
SA WA / WA SA arc-consistent: can we prune any
values?
IT University of Copenhagen

Rune Mller Jensen

14

Arc Consistency Algorithm AC-3

Obs: two arcs for

each binary
constraint!

IT University of Copenhagen

Rune Mller Jensen

15

Complexity of AC-3
Assume
n variables,
at most d values in domains
c binary constraints

Observations
An arc can at most be added d times
An arc can be revised in d2
Thus, worst case runtime is O(cd3)
IT University of Copenhagen

Rune Mller Jensen

16

CSP Solving

IT University of Copenhagen

Rune Mller Jensen

17

Search in CSP
Inference is not enough

Apply depth-first search:

State: Partial assignment
Action: var = value

Complexity
Branching factor b at the top level is nd
b = (n-l)d at depth l, hence n!dn leaves
But only dn complete assignments?!

IT University of Copenhagen

Rune Mller Jensen

18

Backtracking Search
Insight:
If we assign the first k variables to values, it does not
matter in what order we did it in.
Thus, after choosing which variable to assign in a
node, do not change it.

Backtracking Algorithm:
Choose values for one variable at the time.
Backtrack when a variable has no legal values left.

IT University of Copenhagen

Rune Mller Jensen

19

Backtracking Example
NT
WA

IT University of Copenhagen

Rune Mller Jensen

SA

Q
NSW
V
T

20

Backtracking Example
NT
WA

IT University of Copenhagen

Rune Mller Jensen

SA

Q
NSW
V
T

21

Backtracking Example
NT
WA

IT University of Copenhagen

Rune Mller Jensen

SA

Q
NSW
V
T

22

Backtracking Example
NT
WA

IT University of Copenhagen

Rune Mller Jensen

SA

Q
NSW
V
T

23

Backtracking Example
NT
WA

IT University of Copenhagen

Rune Mller Jensen

SA

Q
NSW
V
T

24

Backtracking Example
NT
WA

IT University of Copenhagen

Rune Mller Jensen

SA

Q
NSW
V
T

25

Backtracking Example
NT
WA

IT University of Copenhagen

Rune Mller Jensen

SA

Q
NSW
V
T

26

Backtracking Example
NT
WA

SA

Q
NSW
V
T

IT University of Copenhagen

Rune Mller Jensen

27

Backtracking Algorithm

IT University of Copenhagen

Rune Mller Jensen

28

Improving Backtracking Efficiency

General-purpose methods can give huge gains
in speed:
Which variable should be assigned next?
(SELECT-UNASSIGNED-VARIABLE)
In what order should its values be tried?
(ORDER-DOMAIN-VALUES)
What inferences can be performed
(INFERENCE)

IT University of Copenhagen

Rune Mller Jensen

29

Selecting Variable
NT
WA

SA

NSW
V
T

Minimum remaining values (MRV)

Rule: choose variable with the fewest legal values

IT University of Copenhagen

Rune Mller Jensen

30

Selecting Variable
NT
WA

SA

NSW
V
T

Degree heuristic

Rule: select variable that is involved in the largest

number of constraints on other unassigned variables.
Degree heuristic is very useful as a tie breaker
IT University of Copenhagen

Rune Mller Jensen

31

Selecting Value
NT
WA

SA

NSW
V
T

Least constraining value heuristic

Rule: given a variable choose the least constraining
value i.e., the one that leaves the maximum flexibility
for subsequent variable assignments.
IT University of Copenhagen

Rune Mller Jensen

32

Backtracking Algorithm

IT University of Copenhagen

Rune Mller Jensen

33

Forward Checking
NT
WA

SA

NSW
V
T

Forward checking: Whenever a value v is assigned to a

variable Xi, make all variables consistent with this
assignment.
Terminates search when any variable has no legal
values.
IT University of Copenhagen

Rune Mller Jensen

34

Forward Checking
NT
WA

SA

NSW
V
T

Forward checking: Whenever a value v is assigned to a

variable Xi, make all variables consistent with this
assignment.
Terminates search when any variable has no legal
values.
IT University of Copenhagen

Rune Mller Jensen

35

Forward Checking
NT
WA

SA

NSW
V
T

Forward checking: Whenever a value v is assigned to a

variable Xi, make all variables consistent with this
assignment.
Terminates search when any variable has no legal
values.
IT University of Copenhagen

Rune Mller Jensen

36

Forward Checking
NT
WA

SA

NSW
V
T

Forward checking: Whenever a value v is assigned to a

variable Xi, make all variables consistent with this
assignment.
Terminates search when any variable has no legal
values.
IT University of Copenhagen

Rune Mller Jensen

37

Forward Checking Algorithm

function FORWARD-CHECKING-SEARCH(csp) returns a solution or failure
return RECURSIVE-FORWARD-CHECKING({},csp)
function RECURSIVE-FORWARD-CHECKING(assignment,csp) returns a solution or
failure
if assigment is complete then return assigment
var SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],assigment,csp)
for each value in ORDER-DOMAIN-VALUES(var, assigment, csp) do
if value is consistent with assigment do
add{var=value} to assigment

inferences remove all domain values of remaining variables

inconsistent with {var = value}
if inferencesfailure then
add inferences to assigment and update csp
resultRECURSIVE-FORWARD-CHECKING(assigment, csp)
if resultfailure then
result result
remove {var=value} and inferences from assigment and csp
return failure
IT University of Copenhagen

Rune Mller Jensen

38

Forward Checking
Forward checking propagates information from assigned to
unassigned variables, but doesn't provide early detection for all
failures:
NT
WA

SA

NSW
V

NT and SA cannot both be blue!

IT University of Copenhagen

Rune Mller Jensen

39

Arc Consistency
NT
WA

SA

NSW
V
T

X Y is consistent iff for every value ui of X there is some

allowed vj value in Y

IT University of Copenhagen

Rune Mller Jensen

40

Arc Consistency
NT
WA

SA

NSW
V
T

X Y is consistent iff for every value ui of X there is some

allowed vj value in Y

IT University of Copenhagen

Rune Mller Jensen

41

Arc Consistency
NT
WA

SA

NSW
V
T

X Y is consistent iff for every value ui of X there is some

allowed vj value in Y.
Arc consistency detects failure earlier than FC

IT University of Copenhagen

Rune Mller Jensen

42

MAC
function MAC-SEARCH(csp) returns a solution or failure
run AC-3(csp)
return RECURSIVE-MAC({},csp)
function RECURSIVE-MAC(assignment,csp) returns a solution or failure
if assigment is complete then return assigment
var SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],assigment,csp)
for each value in ORDER-DOMAIN-VALUES(var, assigment, csp) do
if value is consisten with assigment do
add{var=value} to assigment

inferences AC-3(csp)
if inferencesfailure then
add inferences to assigment and update csp
resultRECURSIVE-MAC(assigment, csp)
if resultfailure then
result result
remove {var=value} and inferences from assigment and csp
return failure
IT University of Copenhagen

Rune Mller Jensen

43