Spring 2006 Artificial Intelligence COSC40503 week 7

1
Spring 2006 Artificial Intelligence
COSC 40503week 7

• Antonio Sanchez
• Texas Christian University

2
Heuristic Function

• In a board game, before we make a move when we
  have various
• options How do we know which is the best move?
• Information
• If I have an algorithm that guides to win the
  game (truth)
• Adaptation
• Through learning by trial and error
  (algedonics or backpropagation) knowledge
• Knowledge
• By defining a heuristic function F that give a
  sense of evaluation and comparison between moves,
  allowing me to use a contingent
• IF i THEN k rule selection

3
Heuristic Function

• In a board game, to arrive at that knowledge
• I have to perceive some patterns from the boards,
  this is, aspects that show a comparative value
  among them
• For example, we believe that the green option is
  the best, although this is not an absolute truth,
  nevertheless is knowledge
• IF Black Panel
• THEN Green Panel

• How can we define the knowledge behind this
  opinion?
• After some consideration we may come up with
  something like this
• BoardEvaluation MyOpenOptions -
  TheirOpenOptions
• Using this function we obtain for the three
  possible panels that
• BoardEvaluation(Yellow Board) 8 - 5 3
• BoardEvaluation( Green Board) 8 - 4 4
• BoardEvaluation( Red Board) 8 - 6 2
  BoardEvaluation(Black Board) 8 - 8 0

4
Heuristic Function
E ?aXp

• Let us test the function further on the game with
  another board. Here we have a tie between the
  blue and the green boards. Furthermore we feel
  that the red board is a better one and the
  function is not showing it.
• Our knowledge is incomplete, therefore we must
  add extra terms
• How about

BoardEvaluation MyOpenOptions -
TheirOpenOptions 2 MyWinningOptions Much
better. In general then we can stipulate that a
evaluation function is a polynomial with as many
elements as I think represent the value of the
board. Furthermore each element can by multiply
by factor and elevated to a power to weight the
element in the function E
?aXp Determining the right function is a matter
of expertise i.e. knowledge and commitment
Authors in the 60s Nils Nilseen, A. Samuel,
Herbert Simon y Allen Newell
5
One or many Heuristic Functions

• A final note on the heuristic function. As we
  play further down the game, the function may be
  have to be changed according to where in the game
  we are.
• This is to say at the beginning the strategy of
  the game may be to achieve a given position.
• In the middle of the game maybe the strategy lies
  in capturing pieces.
• Towards the end of the game it maybe that doing
  just one move is important.
• Should this be the case, then we might have
  different X terms with different factors a,p in
  the formula to accommodate the strategy at hand,
  thus having different formulas according a the
  depth of the game

E(t) ?aXp E(tdt) ?aXp E(t2dt)
?aXp
6
Jumping Ahead

• However after I play my move, my opponents will
  play theirs and if they are rational players,
  will use a criteria to select the move in order
  to get a benefit and reducing the value of the
  board just chosen by me.
• Therefore I should evaluate not my next move,
  but a combination of my move and theirs to
  determine my best choice.
• In their case I should consider not a
  maximization of the formula E ?aXp but rather,
  the minimization since by reducing the value of
  the function from their perspective they chooses
  a better board position.
• Lets take a look

7
By the way Bremermanns limit

• Using Einstens E mc2 equation and the
  Heinseberg Uncertainty principle, H. Bremermanns
  derived a limit to the amount of storage capacity
  of matter. It can also be used to determine the
  speed in a communication channel
• The value proposed is 2 1047 bits per second
  per gram. The number is important because it
  gives us a sense of how much can humankind can
  expect to get from nanotechnologies and beyond
• The Ipod stores 15,000 songs in just 5.5 ounces
  represents only a value 3 109 and the smallest
  memory might achieve up to 2 1010 bits per
  gram, yet the limit is there and one day will be
  reached

8
Project 2 and Exam 2

• Note the date (Wed April 12) and requirements
  for project 2
• Mankala the E ?aXp must beat the STM
• Four in a Row the E ?aXp must train the the
  STM after playing no more than 1,000 games. Most
  likely a set of formulas with various degrees of
  difficulty should be used incrementally
• Nim game the E ?aXp must train the the STM,
  letting play it play first. After playing no more
  than 500 games. Most likely a set of formulas
  with various degrees will be necessary
• For all three games the evaluation should be
  done using recursive alpha beta and should do
  well when playing against a human
• Note the date of the next exam
• Friday March 31

9
MiniMax

• The algorithm to evaluate the next move will work
  bottom up as follows
• Branch down two levels (known as one ply)
• Evaluate all the boards
• Minimize by selecting the board with minimum
  value
• Move up one level
• Maximize by selecting the board with the best
  value
• This is the best move
• with a given value from the
• function
• It is called MINImize MAXimize (MINIMAX)

Selected move
5
10
Minimax
3
Max
2
1
3
Min
Eval
11
MiniMax

• Now in reality one should try branch the tree
  further down than one ply, the closer we get to
  the end of the game the closer to the truth our
  knowledge will be.
• As a matter of fact, many computer games that
  you play have generally three levels novice,
  medium, expert. Well the expertise mostly likely
  reflect how far they branch down, most of the
  time we deal with 1,2 and 3 plys. The further
  you go down, the longer it will take to decide.
• Nowadays faster machines are able to go as far
  as down as twelve plys. Imagine just the number
  of boards to evaluate.
• eval gt minimize - maximize - minimize -
  maximize - minimize -maximize gt select

12
Applying minimax

13
Pseudo Code MiniMax
     // Maximize method      int 
Maximize(BOARD,alpha,beta,depthGrow)              
  float max -infinite            for ( j
1  j lt BOARD3   jj1 )                   
MaxBOARD NextBoard(j,BOARD)
POSSIBLE Minimize(MaxBOARD, depthGrow)         
          if POSSIBLE2 gt max BOARD
POSSIBLE                       return BOARD
     

• public class Minimax with alpha beta pruning
• int BOARD new int3  // BOARD1 is the
  board number
• // BOARD2 is the evaluation of the board
• 
  //BOARD3 is the number of possible boards that
  open from this one
• int POSSIBLE new int3 int MinBOARD
  new int3
• int MaxBOARD new int3
• int play, plyLimit, depthGrow // define desired
  board, play and depth 
• //  Recursive MINIMAX
• public void init() 
•     plyLimit   depth    
•      game "playing"
•      while (Game "playing")   
•            
•              BOARD   Maximize(BOARD, 1) 
•              display(BOARD) 
•              game EvaluateEnd(BOARD) 
• if gameplaying BOARD
  otherPlay (BOARD)
• game
  EvaluateEnd(BOARD)

// Minimize Method       int 
Minimize(BOARD,alpha,beta,depthGrow)              
depthGrow depthGrow 1  float min
infinite            for ( j 1  j lt
BOARD3  jj1 )                   MinBOARD
NextBoard(j, BOARD)                 if (
depthGrow plyLimit ) POSSIBLE2
Evaluate(MinBOARD1)                        
else POSSIBLE   Maximize(MinBOARD, depthGrow)
                if ( POSSIBLE2 lt
min  ) BOARD POSSIBLE       
 Return BOARD
// Evaluate method      float Evaluate(int3
BOARD) // determines the value of the
function for the board // Evaluate End
method      String  EvaluateEnd(int3 BOARD)
  // determines if the games has ended //
Display Board      void display(int3 BOARD)   
// display board over the interface // Next
Board Method      int  NextBoard(int j, int3
BOARD)     // determines the next sequential
board    
14
Big trees

• So what do you when you have a tree with many
  branches you do not need
• You prune it, dont you?
• Back in the sixties, A. Samuel did it, he called
  his algorithm
• the a ß pruning
• The idea behind is that the values in the upper
  two nodes already selected might preclude some
  nodes to be considered

15
a ß pruning
gt3
3
does not go
Max
does not go
lt8
goes up as a
lt2
lt1
lt3
goes up and stops (a gt3)
does not go
goes up and stops ! (a gt3)
does not go
Min
does not go
does not go
goes up
goes up as ß
Eval
16
Recursive algorithm

• But the pruning algorithm is recursive
• and goes further down the tree,
• a ß work two levels at a times,
• having stacked values a ß and a ß

a
Much better here ß a values at this
level help in reducing the nodes even way below
ß
a
ß
a ß values at this level help in reducing the
nodes below
17
An example of Alpha-beta pruning
Source www.cs.umbc.edu/ypeng/F02671/lecture-note
s/Ch05.pptt
18
Final tree
max
min
max
min
max

0
5
-3
3
3
-3
0
2
-2
3
Source www.cs.umbc.edu/ypeng/F02671/lecture-note
s/Ch05.pptt
19
Another a ß example
Source www.cs.pitt.edu/milos/
courses/cs2710/lectures/Class8.pdf
20
Pseudo Code a ß
Maximize method      int  Maximize(BOARD,alpha,
beta,depthGrow)                 alpha
-infinite            for ( j 1  j lt BOARD3
  jj1 )                    MaxBOARD
NextBoard(j,BOARD) POSSIBLE
Minimize(MaxBOARD,alpha,beta,depthGrow)          
         if ( beta gt alpha ) alpha
beta                   if alpha gt POSSIBLE2 
then BOARD POSSIBLE                    
   return BOARD      

• public class Minimax with alpha beta pruning
• int BOARD new int3  // BOARD1 is the
  board number
• // BOARD2 is the evaluation of the board
• 
  //BOARD3 is the number of possible boards that
  open from this one
• int POSSIBLE new int3 int MinBOARD
  new int3
• int MaxBOARD new int3
• int play, plyLimit, depthGrow // define desired
  board, play and depth 
• //  Recursive MINIMAX
• public void init() 
•     plyLimit   depth    
•      game "playing"
•      while (Game "playing")   
•            
•              BOARD   Maximize(BOARD,alpha,
  beta, 1) 
•              display(BOARD) 
•              game EvaluateEnd(BOARD) 
• if gameplaying BOARD
  otherPlay (BOARD)
• game
  EvaluateEnd(BOARD)

// Minimize Method       int 
Minimize(BOARD,alpha,beta,depthGrow)              
depthGrow depthGrow 1  pruning "in
process" beta infinite            for ( j
1  j lt BOARD3 AND pruning "in process"  
jj1 )                   MinBOARD
NextBoard(j, BOARD)                 if (
depthGrow plyLimit ) POSSIBLE2
Evaluate(MinBOARD1)                        
else POSSIBLE   Maximize(MinBOARD, alpha, beta,
depthGrow)                 if ( MinBOARD2 lt
beta  )                          BOARD
POSSIBLE   beta MinBOARD2                
if (beta lt alpha)   pruning "done"
                          Return BOARD
// Evaluate method      float Evaluate(int3
BOARD) // determines the value of the
function for the board // Evaluate End
method      String  EvaluateEnd(int3 BOARD)
  // determines if the games has ended //
Display Board      void display(int3 BOARD)   
// display board over the interface // Next
Board Method      int  NextBoard(int j, int3
BOARD)     // determines the next sequential
board    
21
Bigger trees ?

• If you have still a bigger tree you can always
  do a Crash Cut,
• The idea behind this pruning is to cut a
  complete sub tree before branching if you do not
  like a given type of move
• For example, in chess, lets say you do not care
  to exchange your queens, therefore that whole sub
  tree is chopped off

22
Crash Cut algorithm
a
Just take disregard this sub tree
ß
a
ß
23
Pseudo Code Crash Cut
// Maximize with Crash Cut Pruning      int 
Maximize(BOARD,alpha,beta,depthGrow)        
    alpha -infinite                for ( j
1  j lt BOARD3    jj1 )                 
MaxBOARD1 NextBoard(j,BOARD)
                  // funcion dontCut returns
true if there is not a Crash Cut
  If (dontCut(MaxBOARD1)  
                            POSSIBLE
Minimize(MaxBOARD,alpha,beta,depthGrow)
                               if ( beta gt alpha
) alpha beta                               
if alpha gt POSSIBLE2  then BOARD POSSIBLE  
                 return BOARD   
// dontCut Method      boolean dontCut(int
board)     cut true for
(j1 jltcutBoards and cuttrue, jj1)
if board cutListj cut false
return cut

• public class Minimax with alpha beta pruning
• int BOARD new int3  // BOARD1 is the
  board number
• // BOARD2 is the evaluation of the board
• 
  //BOARD3 is the number of possible boards that
  open from this one
• int POSSIBLE new int3 int MinBOARD
  new int3
• int MaxBOARD new int3
• int play, plyLimit, depthGrow // define desired
  board, play and depth 
• //  Recursive MINIMAX
• public void init() 
•     plyLimit   depth    
•      game "playing"
•      while (Game "playing")   
•            
•              BOARD   Maximize(BOARD,alpha,
  beta, 1) 
•              display(BOARD) 
•              game EvaluateEnd(BOARD) 
• if gameplaying BOARD
  otherPlay (BOARD)
• game
  EvaluateEnd(BOARD)

// Evaluate method      float Evaluate(int3
BOARD) // determines the value of the
function for the board // Evaluate End
method      String  EvaluateEnd(int3 BOARD)
  // determines if the games has ended //
Display Board      void display(int3 BOARD)   
// display board over the interface // Next
Board Method      int  NextBoard(int j, int3
BOARD)     // determines the next sequential
board    
// Minimize Method       int 
Minimize(BOARD,alpha,beta,depthGrow)              
depthGrow depthGrow 1  pruning "in
process" beta infinite            for ( j
1  j lt BOARD3 AND pruning "in process"  
jj1 )                   MinBOARD
NextBoard(j, BOARD)                 if (
depthGrow plyLimit ) POSSIBLE2
Evaluate(MinBOARD1)                        
else POSSIBLE   Maximize(MinBOARD, alpha, beta,
depthGrow)                 if ( MinBOARD2 lt
beta  )                          BOARD
POSSIBLE   beta MinBOARD2                
if (beta lt alpha)   pruning "done"
                          Return BOARD
24
Behavior with pruning
25
Game Trees with Chance Nodes

• Chance nodes (shown as circles) represent the
  dice rolls.
• Each chance node has 21 distinct children with a
  probability associated with each.
• We can use minimax to compute the values for the
  MAX and MIN nodes.
• Use expected values for chance nodes.
• For chance nodes over a max node, as in C, we
  compute
• epectimax(C) Sumi(P(di) maxvalue(i))
• For chance nodes over a min node compute
• epectimin(C) Sumi(P(di) minvalue(i))

Min Rolls
Max Rolls
Source www.cs.umbc.edu/ypeng/F02671/lecture-note
s/Ch05.pptt
26
Meaning of the evaluation function
A1 is best move
A2 is best move
2 outcomes with prob .9, .1

• Dealing with probabilities and expected values
  means we have to be careful about the meaning
  of values returned by the static evaluator.
• Note that a relative-order preserving change of
  the values would not change the decision of
  minimax, but could change the decision with
  chance nodes.
• Linear transformations are ok

Source www.cs.umbc.edu/ypeng/F02671/lecture-note
s/Ch05.pptt
27
State of the ArtChess point scale
Source www.cs.umbc.edu/ypeng/F02671/lecture-note
s/Ch05.pptt