Learning the probability of mate in shogi using SVM

1
Learning the probability of mate in shogi using
SVM

• M1 Makoto Miwa
• Chikayama Taura Lab

2
Outline

• Introduction Motivation
• Mate problem
• Whats mate?
• Mate search algorithms
• PDS
• Support Vector Machine
• Solving Linear Problems
• Solving Nonlinear Problems
• Experiments Results
• Conclusion Future Work

3
Outline

• Introduction Motivation
• Mate problem
• Whats mate?
• Mate search algorithms
• PDS
• Support Vector Machine
• Solving Linear Problems
• Solving Nonlinear Problems
• Experiments Results
• Conclusion Future Work

4
Introduction

• Detecting mates is a critical issue in shogi.
• Many successful AND/OR tree search algorithms
• PN, PDS, df-pn
• Almost all computer shogi players have their own
  check-mate-routine.
• In Gekisashi, the check-mate-routine uses about
  30 of the entire search time.

5
Motivation (1)

• Success of machine learning in game playing
• Samuels checkers, TD-GAMMON, Neuro-Go,
• SVM succeeds in many classification problems.
• Little computation required in classification.
• (Although may require much computation in the
  learning phase)

6
Motivation (2)

• Using SVM, can computer game player learn the
  probability of mates as a mated or not
  classification problem?

• With such a classifier, we may be able to
• Control mate-search ordering.
• Control the ratio between mate-search and search
  of other categories.
• etc.

7
Outline

• Introduction Motivation
• Mate problem
• Whats mate?
• Mate search algorithms
• PDS
• Support Vector Machine
• Solving Linear Problems
• Solving Nonlinear Problems
• Experiments Results
• Conclusion Future Work

8
What is mate?

• Mate in the narrow sense
• Opponent king will be captured in the next turn.
• Mate in the wide sense
• Opponent king will eventually be mated in the
  narrow sense.
• ex.
• artificially constructed mating problems

9
Mate Search Algorithms

• Purpose
• Judge whether a position is mated or not.
• If mated, find a way to mate.
• Method
• Many successful AND/OR tree search algorithms.
• PN, PDS, df-pn

10
PDS A. Nagai et al., 1998 (1)

• Proof-number and Disproof-number Search
• Proof-number (pn)
• The least number of leaf nodes that must be
  evaluated true to prove the node is true.
• Disproof-number (dn)
• The least number of leaf nodes that must be
  evaluated false to prove the node is false.
• Depth-first search with multiple
  iterative-deepening.
• PDS uses a transposition table to store
  information on search already made.
• Like best-first search, PDS searches a certain
  node with smallest (dis)proof number at (AND)OR
  node.

11
PDS A. Nagai et al., 1998 (2)

• Set two thresholds, (pn,dn) to 1.
• Expand a certain node with smallest (dis)proof
  number at (AND)OR node updating the parent pn
  dn, while the pn and dn dont exceed the
  thresholds.
• If pn or dn of the root node is set to 0, search
  ends.
• If PDS fails to expand the root node, it increase
  the threshold and search again.

7,10
th(7, 10)
7,3
9,3
7,4
2,3
1,3
4,5
pn,dn
OR node
AND node
pn,dn
12
PDS A. Nagai et al., 1998 (2)

• Set two thresholds, (pn,dn) to 1.
• Expand a certain node with smallest (dis)proof
  number at (AND)OR node updating the parent pn
  dn, while the pn and dn dont exceed the
  thresholds.
• If pn or dn of the root node is set to 0, search
  ends.
• If PDS fails to expand the root node, it increase
  the threshold and search again.

th(8, 10)
7,10
Increase the threshold.
pn,dn
OR node
AND node
pn,dn
13
PDS A. Nagai et al., 1998 (2)

• Set two thresholds, (pn,dn) to 1.
• Expand a certain node with smallest (dis)proof
  number at (AND)OR node updating the parent pn
  dn, while the pn and dn dont exceed the
  thresholds.
• If pn or dn of the root node is set to 0, search
  ends.
• If PDS fails to expand the root node, it increase
  the threshold and search again.

th(8, 10)
7,10
pn,dn
OR node
AND node
pn,dn
14
PDS A. Nagai et al., 1998 (2)

• Set two thresholds, (pn,dn) to 1.
• Expand a certain node with smallest (dis)proof
  number at (AND)OR node updating the parent pn
  dn, while the pn and dn dont exceed the
  thresholds.
• If pn or dn of the root node is set to 0, search
  ends.
• If PDS fails to expand the root node, it increase
  the threshold and search again.

th(8, 10)
7,10
7,3
9,3
7,4
th(8, 3)
Look up the transposition table
pn,dn
OR node
AND node
pn,dn
15
PDS A. Nagai et al., 1998 (2)

• Set two thresholds, (pn,dn) to 1.
• Expand a certain node with smallest (dis)proof
  number at (AND)OR node updating the parent pn
  dn, while the pn and dn dont exceed the
  thresholds.
• If pn or dn of the root node is set to 0, search
  ends.
• If PDS fails to expand the root node, it increase
  the threshold and search again.

th(8, 10)
7,10
7,3
9,3
7,4
th(8, 3)
2,3
1,3
4,5
th(2, 3)
pn,dn
OR node
AND node
pn,dn
Look up the transposition table
16
PDS A. Nagai et al., 1998 (2)

• Set two thresholds, (pn,dn) to 1.
• Expand a certain node with smallest (dis)proof
  number at (AND)OR node updating the parent pn
  dn, while the pn and dn dont exceed the
  thresholds.
• If pn or dn of the root node is set to 0, search
  ends.
• If PDS fails to expand the root node, it increase
  the threshold and search again.

th(8, 10)
7,10
7,3
9,3
7,4
th(8, 3)
2,3
0,8
4,5
pn,dn
OR node
AND node
pn,dn
Proven to be mated
17
PDS A. Nagai et al., 1998 (2)

• Set two thresholds, (pn,dn) to 1.
• Expand a certain node with smallest (dis)proof
  number at (AND)OR node updating the parent pn
  dn, while the pn and dn dont exceed the
  thresholds.
• If pn or dn of the root node is set to 0, search
  ends.
• If PDS fails to expand the root node, it increase
  the threshold and search again.

th(8, 10)
7,10
7,3
9,3
7,4
th(8, 3)
2,3
0,8
4,5
th(4, 3)
pn,dn
OR node
AND node
pn,dn
18
PDS A. Nagai et al., 1998 (2)

• Set two thresholds, (pn,dn) to 1.
• Expand a certain node with smallest (dis)proof
  number at (AND)OR node updating the parent pn
  dn, while the pn and dn dont exceed the
  thresholds.
• If pn or dn of the root node is set to 0, search
  ends.
• If PDS fails to expand the root node, it increase
  the threshold and search again.

th(8, 10)
7,10
7,3
9,3
7,4
th(8, 3)
2,3
0,8
4,5
th(4, 3)
th(3, 3)
pn,dn
OR node
AND node
pn,dn
th(4, 3)
Multiple iterative deepening
19
PDS A. Nagai et al., 1998 (2)

• Set two thresholds, (pn,dn) to 1.
• Expand a certain node with smallest (dis)proof
  number at (AND)OR node updating the parent pn
  dn, while the pn and dn dont exceed the
  thresholds.
• If pn or dn of the root node is set to 0, search
  ends.
• If PDS fails to expand the root node, it increase
  the threshold and search again.

th(8, 10)
0,8
0,8
9,3
7,4
th(8, 3)
0,8
0,8
0,8
pn,dn
OR node
AND node
pn,dn
20
PDS A. Nagai et al., 1998 (3)

• How do we use PDS in shogi?
• search(depth)
• if(depth 0)
• return eval()
• PDS(pos, depth)
• search(depth 1)

• To find a position not-mated is relatively
  difficult.
• It is not clear whether the best search based
  on the criterion of pn and dn is real best
  search or not.

21
Outline

• Introduction Motivation
• Mate problem
• Whats mate?
• Mate search algorithms
• PDS
• Support Vector Machine
• Solving Linear Problems
• Solving Nonlinear Problems
• Experiments Results
• Conclusion Future Work

22
Support Vector Machine (1)

• Proposed by Vapnik et al. in 1992
• Two-class classifier
• Important Features
• Maximize margin
• avoid overfitting
• Nonlinear mapping
• deal with linearly non-separable data
• Kernel function
• simplify inner product (dot product) computation
• Support vectors
• represent a hyperplane with a small set of input
  data

23
Support Vector Machine (2)

• Purpose
• Sort out data into two classes.
• Method
• Construct the hyperplane that maximize the
  margin.
• Find support vectors that represent the
  hyperplane.

24
SVM - Solving Linear Problems - (1)
Constructs the hyperplane (h) that maximize the
margin.
Problem
Margin
h
dual problem
Support Vector
25
SVM - Solving Linear Problems - (2)
Problem
Solution
We can make classifiers with support vectors only.
Classifier
All computations with input data are dot products.
26
SVM Solving Nonlinear Problems (1)

• Soft Margin Algorithm
• Use relaxed separation constraints.

h
(C-SVC)
Support Vector
27
SVM Solving Nonlinear Problems (2)

• Kernel Trick
• Mapping vectors to a higher-dimension space
  (Feature Space).

Map to a higher-dimension space
28
SVM Solving Nonlinear Problems (3)

• Kernel Trick
• Mapping vectors to a higher-dimension space.
• Kernel Function
• SVM uses only dot products with input data.
• SVM doesnt need to calculate in higher
  dimensions.
• ex.
• Polynomial
• RBF
• Sigmoid

29
Outline

• Introduction Motivation
• Mate problem
• Whats mate?
• Mate search algorithms
• PDS
• Support Vector Machine
• Solving Linear Problems
• Solving Nonlinear Problems
• Experiments Results
• Conclusion Future Work

30
Motivation

• Using SVM, can computer game player learn the
  probability of mates as a mated or not
  classification problem?

• Experiments
• Measure the accuracy of classification.
• Use the obtained information to control
  mate-search ordering.

31
Experiments - Classifier - (1)

• Purpose
• Measure the accuracy of classification
• Define mate as
• A mate can be found with the PDS algorithm by
  inspecting not more than 100,000 nodes.
• One mate can be found in 1 sec.
• Fast enough for practical use.

32
Experiments - Classifier - (2)

• SVM
• LIBSVM
• SVM Library developed by Chih-Chung Chang and
  Chih-Jen Lin
• http//www.csie.ntu.edu.tw/cjlin/libsvm/
• C-SVC
• Kernel Function
• Linear
• RBF(Radial basis function)

33
Experiments - Classifier - (3)

• Features Considered (for each sides) 361
  features
• Features about the king
• Position of the king, distance between two kings
• Features concerning 24 squares around the king
• At a board edge, occupied or not, square control
• Number of squares the king can escape to
• Number of defense pieces, attack pieces and
  pinned pieces
• Pieces in hand
• Which pieces and how many of them are in hand
• Potential attack
• Pieces can be put on or moved to a square from
  which they can attack the king.

34
Experiments - Classifier - (4)

• Datasets Usage
• Divide 694 end-game positions played by Habu
  against other professionals into 3 sets. For each
  set, 10,000 positions are randomly chosen from
  those positions visited in search starting from
  the end-game positions belonging to the set.
• 1 train set (10000 examples) and 1 test set
  (20000 examples).
• 1 train set (20000 examples) and 1 test set
  (10000 examples).
• 139 artificially constructed mating problems as
  an independent test set.

35
Experiments - Classifier - (6)

• Experiment environment

36
Result - Classifier - (1)

• Accuracy

37
Result - Classifier - (2)

• Computation time

Faster than PDS search. (0.49 msec vs 2.0 msec)
38
Experiment - Search controller - (1)

• Use the classifier in search order control of
  PDS.
• Use the distance between a position and the
  classification hyperplane as the probability of
  mate. (Proposed by Platt.)
• Used to order nodes whose proof- and
  disproof-numbers are the same.

39
Experiment - Search controller - (2)

• Train data set
• 10,000 positions randomly chosen from those
  positions visited in search starting from 694
  endgame positions played by Habu against other
  professionals.
• Test data set
• 694 endgame positions
• 139 artificially constructed mating problems as
  an independent test set.
• SVM
• C-SVC with linear-kernel.

40
Result - Search controller -
Sorting vs no sorting
41
Outline

• Introduction Motivation
• Mate problem
• Whats mate?
• Mate search algorithms
• PDS
• Support Vector Machine
• Solving Linear Problems
• Solving Nonlinear Problems
• Experiments Results
• Conclusion Future Work

42
Conclusion

• The classifier shows decent accuracy.
• Larger training sets result in higher accurasy.
• With the linear kernel, a classification is fast
  enough for practical use.
• The classifier did not improve the performance of
  PDS much.

43
Future Work

• Find an appropriate feature set. To analyze
  feature, use linear-kernel results and results of
  some other algorithms, such as neural nets.
• Optimize SVM parameters using SVC and so on.
• Increase meaningful training data.
• Try adopting the classifier in control of other
  search algorithms, such as df-pn or best first.
• Use as the weight controller between mate-search
  and search of other categories.
• Compare with other classifiers such as NN,
  Boosting and so on.
• Now implementing Back Propagation.