AIMA Code: Adversarial Search

1
AIMA Code Adversarial Search

• CSE-391 Artificial IntelligenceUniversity of
  Pennsylvania
• Matt Huenerfauth
• February 2005

2
Game Class

• class Game
• . . .
• def terminal_test(self, state)
• "Return True if this is a final state for
  the game.
• def utility(self, state, player)
• "Return the value of this final state to
  player."
• def to_move(self, state)
• "Return the player whose move it is in
  this state.
• def successors(self, state)
• "Return a list of legal (move, state)
  pairs."

3
MINIMAX

• Search a game tree all the way to the end.
• Start at initial game state.
• Exhaustively expand children nodes (all the
  possible moves in the game from each board).
• When you hit an end state, decide if it was
  good/win or bad/loss and give it an appropriate
  value.
• Then pass up the MIN or the MAX of the values of
  the children at each level (depending on whos
  turn it was).
• At the top level, make the best move follow the
  child that passes up the best value.

4
argmax(list,func) see utils.py

• def argmax(list, func)
• "Return element of list with highest
  func(listi) score"
• best list0 best_score func(best)
• for x in list
• x_score func(x)
• if x_score gt best_score
• best, best_score x, x_score
• return best

5
Use of argmax(list,func)

• Well see this line of code on the next slide
• state will be a state of the game
• call_min is a function well define later
• argmax( game.successors(state),
• lambda ((a, s)) call_min(s) )
• Successors is a list of children of this state
  (action, state) pairs.
• The lambda function created takes two arguments.
  
• It ignores the first and passes the second to
  call_min().
• Applied to (action, state) pairs, it calls
  call_min on each state.
• Argmax returns pair with highest value for
  call_min(state).

6

• At the root of the MINIMAX search at some state
  in a game.
• def minimax_decision(state, game) In a state
  in a game.
• player game.to_move(state) Whos turn
  is it?
• action, state \
• argmax(game.successors(state), What
  successor returns lambda ((a, s))
  call_min(s)) maximum call_min()?
• return action Move to that state.
• We first decide whose turn it is.
• Then our function will tell us what move to make
  that will move us to the successor state that-
  has the MAXIMUM return value - when we call the
  CALL_MIN function on it.
• Were going to see a trend here pretty soon

7

• Take the MAX of your MINs,
• and the MIN of your MAXs

8

• def call_max(state)
• if game.terminal_test(state) Check if
  game over.
• return game.utility(state, player)
• v -infinity
• for (a, s) in game.successors(state)
  Return the maximum
• v max(v, call_min(s)) of the
  call_mins.
• return v
• def call_min(state)
• if game.terminal_test(state) Check if
  game over.
• return game.utility(state, player)
• v infinity
• for (a, s) in game.successors(state)
  Return the minimum
• v min(v, call_max(s)) of the
  call_maxs.
• return v

9
Initial call to minimax_decision

• def call_max(state)
• if game.terminal_test(state)
• return game.utility(state, player)
• v -infinity
• for (a, s) in game.successors(state)
• v max(v, call_min(s))
• return v
• def call_min(state)
• if game.terminal_test(state)
• return game.utility(state, player)
• v infinity
• for (a, s) in game.successors(state)
• v min(v, call_max(s))
• return v

GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
10
Depth Limited MINIMAX

• Most games are so complex that searching all of
  the possible moves in the game until you reach
  GAME OVER on every branch takes too long.
• So, we set a limit on how far down to look.
• Now, well need a function that can look at a
  game state thats not yet finished. It needs to
  guess how good it thinks that state of the game
  is.
• As we discussed in class, an Evaluation
  Function looks at non-terminal state, and
  returns a value of how promising that state looks.

11
Depth Limited MINIMAX

• So, we add a depth limit. Each time we make a
  call to the next level down, well decrement this
  depth limit.
• def minimax_decision(state, game, d) Were in
  some state during a game.
• player game.to_move(state) Whos turn
  is it?
• action, state \ Return the
  successor of current argmax(game.successors(st
  ate), board that returns the highest
• lambda ((a, s)) call_min(s, d-1))
  value when we call the
• return action call_min function on
  it.

12

• def call_max(state, d)
• if d 0 Use eval. func. if hit
  limit.
• return game.evaluation_function(state,
  player)
• if game.terminal_test(state) Check if
  game over.
• return game.utility(state, player)
• v -infinity
• for (a, s) in game.successors(state)
  Return the maximum
• v max(v, call_min(s, d-1)) of the
  call_mins.
• return v
• def call_min(state, d)
• if d 0 Use eval. func. if
  hit limit.
• return game.evaluation_function(state,
  player)
• if game.terminal_test(state) Check if
  game over.
• return game.utility(state, player)
• v infinity
• for (a, s) in game.successors(state)
  Return the minimum
• v min(v, call_max(s, d-1)) of the
  call_maxs.
• return v

13
Initial call to minimax_decision

• call_max
• call_min

GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
14
Initial call to minimax_decision

• call_max
• call_min

GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
GAMEOVER
EVALUATIONFUNCTION!!!
15
Alpha Beta Search

• Doing a little pruning

16
Alpha Beta

• Think about this momentof the search process

23
16 23 5
Imagine we already looked at all the stuff down
this part of tree
24
Do we even need tobother looking at thesetwo
last red ones here?Could their blue parentever
get picked up top?
17
Alpha Beta

• The blue node will return at least a 24.So,
  its branch willnever get picked bythe
  top-level node.

23
? 24
16 23 5
Imagine we already looked at all the stuff down
this part of tree
24
Do we even need tobother looking at thesetwo
last red ones here?Could their blue parentever
get picked up top?
18
Alpha Beta

• So, dont bother lookingat the blue nodes
  otherchildren. They just dontmatter.

23
? 24
16 23 5
Imagine we already looked at all the stuff down
this part of tree
24
19
Alpha Beta

• Continue with the rest of the search.

23
? 24
16 23 5
Imagine we already looked at all the stuff down
this part of tree
24
20
Alpha Beta Search

• A version of MINIMAX that incorporates this handy
  trick.
• It lets us prune branches from our tree.
• This saves us a lot of searching through useless
  parts of the tree.
• How do we modify the MINIMAX code to do this?
  (For now, lets forget about the depth limit
  stuff easy to add later.)

21
Implementing Alpha Beta

• First Step Thread alpha and beta values through
  all the calls
• Alpha highest value seen in any call_max on the
  path from root to the current node.
• Beta lowest value seen in any call_min on the
  path from root to current node.

22

• def call_max(state, alpha, beta)
• if game.terminal_test(state) return
  game.utility(state, player)
• v -infinity
• for (a, s) in game.successors(state)
• v max(v, call_min(s, alpha, beta))
• if v gt beta return v stop checking
  kids if any ? beta
• alpha max(alpha, v) alpha highest
  value seen in a max
• return v no pruning trick if we got
  to here
• def call_min(state, alpha, beta)
• if game.terminal_test(state) return
  game.utility(state, player)
• v infinity
• for (a, s) in game.successors(state)
• v min(v, call_max(s, alpha, beta))
• if v lt alpha return v stop checking
  kids if any ? alpha
• beta min(beta, v) beta lowest
  value seen in a min
• return v no pruning trick if we got
  to here

23
At the root

• def alpha_beta(state, game)
• player game.to_move(state)
• action, state \
  argmax(game.successors(state),
• lambda ((a, s)) call_min(s, -infinity,
  infinity))
• return action
• Alpha Starts at infinity so we know it will get
  set to new value first time its maxed inside a
  call_max function.
• Beta Starts at infinity. Same reason for min.

24
Wrap Up

• Game Class
• Utility(), Evaluation_Function(), Is_Move()
• MINIMAX
• Search the whole game tree to the end each time.
• Depth-Limit MINIMAX
• Stop before hitting GAME OVER in all branches.
• Alpha-Beta
• Cool pruning trick to eliminate useless parts of
  tree.