Lecture 17 Influence Maps

CSci 4511 Intro to AI

3.21.11
baylor
University of Minnesota

We begin our story with a

bunch of problems

Terrain Reasoning
Old fashioned war
Strategies:
Strengthen front
Attack flank
Pincer

Ambush rear

Terrain Reasoning
Old fashioned war
Strategies:
Strengthen front
Attack flank
Pincer

Ambush rear

But where are

those?

Problem 1. Finding Flanks

Old fashioned war
Strategies:
Strengthen front
Attack flank
Pincer

Ambush rear

But where are

those?
Group units close to each other

Problem 1. Finding Flanks (contd.)

Old fashioned war
Strategies:
Strengthen front
Attack flank
Pincer

Ambush rear

But where are

those?
Draw line through cores
(so can tell whos behind or side)

Problem 1. Finding Flanks (contd.)

Old fashioned war
Strategies:

Pincer

Flank
Fr
on
t

Strengthen front
Attack flank
Ambush rear

But where are

those?

Flank

Rear

Perpendicular line is the Front

Sides = Flanks, Behind = Rear

Problem 1. Finding Flanks (contd.)

Old fashioned war
Strategies:

Pincer

Flank
Fr
on
t

Strengthen front
Attack flank
Ambush rear

But where are

those?

Flank

Rear

How do we make a computer

understand that?

MOUT
Dont get to play tanks and trenches much
anymore
Stupid nuclear bombs

MOUT Military Operation in Urban Terrain

Cities
Filled with hiding spots
And civilians
And enemies who look like civilians
We care because of the CNN effect. And stuff

Modern training is for individual combat, nonlethal combat, checkpoints, etc

Problem 2. City Defense

Where do we put:
Snipers
Sentries
Sentry guns
Heavy weapons
Traps
Ambush

stairs,
from water

bridge w.
1m / 3ft walls

wall,
1m / 3ft

building,
1 door,
2 windows

Image courtesy of William van der Sterren

deep

wall,

water

2m / 7ft

Problem 2. City Defense

Bad for Sniper
long range view
but exposed

?
Checkpoint
chokepoint

Sniper
long range view,
enclosed space,

stairs,
from water

bridge w.
1m / 3ft walls

wall,
1m / 3ft

building,
1 door,
2 windows

We can see this but how can a computer see it?

deep

wall,

water

2m / 7ft

Terrain Reasoning
These problems are instances of terrain
reasoning
Computers cant see
Sees raw data, not groupings, relationships
Humans very good at this

Understanding the space is important

Problem 3. Path Finding

Plenty of algorithms for finding paths

Path Finding (contd.)

Plenty of things should influence path
Some are semi-permanent
Terrain type
Light (for games where you sneak around)
Sound (ditto)

Some are dynamic

Enemies
Buildable/destroyable walls (RTSes, Tower Defense,
Red Faction)
Fog of War
Treasure
Mobile Targets (e.g., ball in soccer)

Problem 3. Path Finding

Plenty of algorithms for finding paths

But they dont take into account things like risk

Building
Several game types require building
Most obviously Real Time Strategies (RTSes)

Factors
Place walls where they safeguard the most /
most important items
Place buildings near defense
Place buildings near resources

Placing farms near each other

make them easier to defend
with walls, nearby troops

Problem 4. Building location

Building
Several game types require building
Most obviously Real Time Strategies (RTSes)

Factors
Place walls where they safeguard the most /
most important items
Place buildings near defense
Place buildings near resources

Also, Area of Effect (AoE) attacks

Placing farms near each other

make them easier to defend
with walls, nearby troops

but also make them

more vulnerable to
AoE attacks
Problem 5. Bomb placement

Building (contd.)
Lots of games need to be able to place things in
a good place
So they need to understand their environment

Includes types that havent been made yet

Urban renewal games

Lemonade stands
Parties (e.g., Hello Kitty Party Pals)
Shopping games (PaperDoll Tower Defense)
Synergy games
Clownfish & Anemones, Cinnabon & Weight Watchers

6. Ever-Popular Real Estate Game!

We want to build a
house
Close to roads good
Elves are good, but
not too close
Dont build near
haunted castles

Where should we
build?

How Do We Solve These?

7. One More Example

Should we attack that army?
Lots more of
Them

Them

Us

Note fancy big words!

Attenuated Threat Radius

Assume each unit is worth a certain
amount (danger/attack strength)
Each has a threat radius
Represents reach of unit (if ranged)
Also spots unit can reach quickly

Threat drops off with distance

Red Tank
Base Value: 10
Propagation Divisor: 2
Propagation Distance: 3

7. One More Example

Add unit values to map, Sum values
Lots more of
Them

Them

Us

7. One More Example

Should we attack that army?
Lots more of
Them

One of
ours /

Them

Us

7. One More Example

Quantitative
representation
Computer can
reason with this

Computationally
simple and fast
Just adds and
multiplies
Linear growth in
computation time

8. Reminder: Go is Hard
for computers

10761 games
vs. 1040 vs. 10120

Champion: Not Yet

Brute Force does
not scale very well

Influence Maps
Created in 1969 by Albert Zobrist
Computers didnt understand spatial patterns
This capability is important to human problem solving

Looking to capture human-like visual organization

Divide the world into a grid

Each piece adds a number to the grid
Values flow through the grid
Like oil/water (borders) or electric charge (intensity)

Add it all up and you understand the world

Note: You still have to decide how to use the info

Multiple Maps
Can have more than one type of influence
Represent different concerns / factors
Can use all of them at the same time
Sum across influence types / maps

Can select relevant maps at run time

Can be specific to goal, agent, state, personality, etc.

All started with Go

Goals: Pieces grouped into armies, influence of
armies, segmentation of board
Maps: Borders, # empty squares in segment (safety),
breathing spaces in chain, # stones in chain, #
neighbors
Zobrist, A. L., "A model of visual organization for the game of GO," AFIPS Conf. Proc., 1969

Anything You Want

In GO
Influence
Concentration
Tension
Instability

For an RTS
Borders
Resources
Defensibility
Line of fire
Fog of war

Types of Influence Maps

Passability
Resources
Exploration
Threat
Light level
Historical
Emotion (fear, beauty, etc.)
Anything you make up
Desirability Map
Summation of the above

Light Map, Action Quake 2

Propagation
Can propagate however you want
Predefined actor halo, rules
Auto-propagation - linear, exponential, Gaussian, etc.

Can have different conductance for each tile

Walls or mountains could block
Roads or supply lines might spread influence more

Roads
spread
influence
Lakes
block it
(for tanks)

Problems Revisited

6. Ever-Popular Real Estate Game!

Road that looks like water

Elf

Evil
Castle

Place A Grid Over The Map

Convert Map Into Something Easy For

Us To Reason With

O
R
O
O
O

O
R
O
X
O

O
R
O
R
O

O
O
R
O
O

O
O
E
R
R

Convert Grid To Numbers

Normal land is worth 0

Road is worth 1

Live close to elves, but not too close

-1

-1

-1

-1

-1

Stay away from evil castles

-1

-1

-1

-1

-2

-2

-2

-1

-2

-1

-2

-1

-2
-2

-2

Each Object Affects The Area Around It

Squares next to
castle worth -2

-1

-1

-1

-1

-2

-2

-2

-1

-2
-2

-2

-2

-1

-2

-1

Squares next to
those squares
worth -1

Add The Values Together

-2

-1

-2

-2

-1

-1

-2

-1

-2
-2

-2

We Build At The Highest Number

-2

-1

-2

-2

-1

-1

-2

-2

0
Right here

-2

-2

-1

We Build At The Highest Number

Right here

Placing farms near each other

make them easier to defend
with walls, nearby troops

but also make them

more vulnerable to
AoE attacks
Problem 4. Building location

Problem 5. Bomb placement

A* runs a lot
faster when
you do it this
way

Shortest distance doesnt

use cover, good way to get
shot
Problem 3. Path Finding

Note:
Color, like
danger,
fades with
distance

A lot more
work for A*

Waypoints are marked as

being visible to the enemy,
factors into the node cost

Problem 1. Finding Flanks

Old fashioned war
Strategies:

Pincer

Flank
Fr
on
t

Strengthen front
Attack flank
Ambush rear

Flank

Rear

How a human sees the situation

0 = border / no mans land

Ferretman, AI Game Programming Wisdom volume 1

Problem 2. City Defense

Bad for Sniper
long range view
but exposed

?
Checkpoint
chokepoint

Sniper
long range view,
enclosed space,

stairs,
from water

bridge w.
1m / 3ft walls

wall,
1m / 3ft

building,
1 door,
2 windows

We can see this but how can a computer see it?

deep

wall,

water

2m / 7ft

What Makes A Good Sniping Spot?

Inconspicuous
Overlooking distant locations
Overlooking key locations
Target locations lack cover
Hard to reach from target locations
Nearby cover for sniper
Hard to surprise
Flank protected
Have freedom of movement
Focus

Data We Need

Line of sight
Travel time from one point to another
Distance from one point to another
Which part of area do we belong to
Are we in a specific room, courtyard, etc.

Case: Tactical Assessment

Waypoint relations
(potential) interaction or influence
between two waypoints
directional, leveled [0..1]
Examples
nearby cover from threat
hard to reach from target
overlooking key locations
hard to surprise

Case: Tactical Assessment

Relation: nearby cover from threat
the amount of nearby cover
from threat and nearby positions

weapon performance

1.5
1
0.5
0
0

100

1.0

0.5

2/9 = 0.22
0.0

protection

200

Case: Tactical Assessment

nearby
cover
from
seen
threats
east of
position

Case: Tactical Assessment

Focus
concentration of relations in certain
direction (sphere sector)
If long LoS in one direction but
not another, focus is high
If long LoS in all directions,
no focus

Useful for determining flank protection

No focus

Partial focus (blocked 3 directions, open 3 directions)

Case: Tactical Assessment

focus
east

(E,NE,SE, Eup, Edown)

Case: Tactical Assessment

local
property:
light level

Terrain Characteristics Used

Technical Notes (so far)

Williams formula for CGF for AQ2
17 ingredients
Checks in 26 directions

Specifics for this example (map)

480 waypoints
10 seconds to calculate on a PIII/500
Done off-line

24KB of sniping ratings

Making Your Own Formula

How William created the CGF for AQ2
formula
10% Analysis
40% Experimenting
50% Tuning

Adding History
Store Sniping behavior at each waypoint
Time spent sniping
Time spent at location
Time spent zoomed in
Shots fired

Damage done while sniping

Damage taken while sniping

Breakdown by
Sniping spot
Weapon used
Direction faced

Case: Tactical Assessment

positive
sniping
experience,
after
40 rounds,
non-directional

Case: Tactical Assessment

sniping
east,
using
geometry
+ physics
+ learning

Case: Tactical Assessment

traffic
data
(all teams)
after
40 rounds

Case: Tactical Assessment

sniping
east,
using
geometry
+ physics
+ learning
+ tactical
game play
analysis

Case: Tactical Assessment

Results

sniping activity
tracking [ 0.0s]

sniper spot
analysis [~20 s]

level design

physics

waypoints

paths +
line of
sight / fire travel time

terrain analysis

game round
game round
game round

sniper
activity
traffic +
kills ...

reinforcement
learning [0.1s]

analyse activity
+ rerank locations

sniper
activity
sniping
ratings

Bonus Problems!

9. Track and Predict

Target runs behind something and now we
cant see him
Want to predict where hes going to pop
out

9. Track and Predict

house

door

threat

window

Situation (top view)

AI actor
Slides stolen directly from William

9. Track and Predict

?
Where to look for the threat?
What to aim for?

9. Track and Predict

Option:
assume threat static
after moving out of sight
(often incorrect)

!?

9. Track and Predict

Option:
assume threat continues
last observed movement
(incorrect after a little while)

!?

9. Track and Predict

The waypoint grid

(overlay)

9. Track and Predict

The waypoint grid

(AI view)

9. Track and Predict

Extrapolate movement along waypoints

(valid paths, movement speed known).
Threat no longer leaves the house!

9. Track and Predict

Use known line-of-sight for negative

observations.
Exclude some likely threat movements.

9. Track and Predict

How can we use the waypoint graph?
Restrict guesses to valid movements
Estimate the threats movement speed
Exclude visible potential movements
that were not observed

Predict a likely path

Path best aligned with last observed direction

Using per waypoint traffic statistics
Towards most attractive nearby objective
Towards strongest defensive position

WWiD (unrelated to terrain reasoning)

Could ask what would i do in their situation?

9. Track and Predict

Where to aim?
Where you think the threat will likely reappear

Good guesses
Visible spot closest to threat
First visible spot on predicted path
If you think hes coming back for you
First visible spot on shortest path back to you

Any nearby portal (door, window, hole, ...)

9. Track and Predict

3

2
1

Threat turns to attack (1) or

moves to nearest attack position
(2), or continues movement (3)

10. Grenade Handling

Cant (or dont want to) shoot at enemies?
Throw grenades!
Best part grenades bounce!
Humans good at this, AI bad
You do not want to compute this at run-time
If youre going to rebound, there are sooo
many places you can throw
Physics are imprecise so you gotta Monaco it

Precompute, store results in terrain map

Mark the good spots in your map

Information
to track to
determine
value of spot
as a grenade
tossing spot

AI is much
easier with
good
debugging
tools

Have no direct shot at enemy. But we could throw a grenade from the green spot

We know we can hit this because we pre-computed it long before the game began

Hey, howd he do that?

Grenades: A Warning
To throw grenades, need to know
Where to stand
Point to aim at

Grenades and other bouncy things require

you to be very precise
Clean ray trace is not good enough

If you are just a little off, very bad things

can happen

good

bad
oops, we
clipped
the wall

Its very important to be very precise. Or very bad things could happen

11. Team Maneuvers

Choosing an avenue of approach
Bounding overwatch
One person moves, the other covers him

Keeping clear lines of fire

Keeping to cover
Flanking

11. Team Maneuver

how to
attack the
threats as
a team?

1
2

11. Team Maneuver

Approach
Initially, sequentially for each member
evaluate nearby waypoints (within r )
for position improvement
claim best position p
move up
to position p

Repeat on
member
initiative

Things To Evaluate For Each Waypoint

Travel time to / from threat, power-up, ...

Nearby cover and flank protection
Tactical rating of position
Min projected distance to team member
Spread out a bit

Keep out of friendly line-of-fire

Maintain visual contact with enemy

11. Team Maneuver

how to
attack the
threats as
a team?

1
2

11. Team Maneuver

evaluate for
each waypoint:
-distance to
threat
-spread
-nearby cover
-not claimed
-etc.
and claim
best position

1
2

11. Team Maneuver

member1
claims position
and will move up

1
3
2

11. Team Maneuver

member2
evaluates
and picks
best position
given
existing and
claimed
positions

1
3

11. Team Maneuver

same for
member3
however, in
the search
radius, the
best spot
might not be
that good
(LOF blocked)

1
3

11. Team Maneuver

upon arrival,
member3
indeed sees
a blocked
line-of-fire
he quickly
executes
the same
procedure
from his
new spot

!#%&!

1
3

11. Team Maneuver

now member3
moves up
again to
a good position
in an obstacle
rich environment,
youll get flanking
for free

Questions?