Vision based rock, paper, scissors game

1
Vision based rock, paper, scissors game

• Patrik Malm
• Standa Mike
• József Németh
• István Vincze

2
Task

• Create a vision based rock, paper, scissor game
• Realtime webcam image-processing

3
Flowchart
4
Initiation

• Smooth the image
• Color-space conversion - Luv
• The background is captured so that it can be
  subtracted in the following segmentation

5
Hand segmentation

• Normal Bayesian classifier
• Mathematical morphology to decrease noise
• Use labeling to find the largest coherent
  component (or the two largest components in the 2
  player game)
• Cut the hand at the wrist

6
Isolating the hand

• Cut the arm where the size is 70 of the maximum
  projection value
• It works best for arms with vertical orientation
• Improved solution using PCA which works for
  rotated arms as well

7
Isolating the hand

• The approximation of hand rotation is done using
  principal component analysis (PCA)
• It is computed using eigenvectors of the
  second-order moment matrix (covariance matrix)

8
Hand tracking

• Find the centre point of the hand in each frame
• Analyze the movement by looking at the relative
  change in position over a few frames
• The hand tracking is used to countdown

9
Gesture identification

• Logarithmic values of three of the Hu set of
  invariant moments
• Bayesian classifier which then determines the
  current hand gesture

10
Computer player

• Computer choose randomly (current)
• Markov chain to recognize the human opponents
  game strategy (implemented but not added)

11
Known issues

• Skin color variations
• Sensitivity to lighting
• Background artifacts (skin or skin-like colored
  objects)

12
Thank you for your attention

• Bibliography
• A.A. Argyros, M.I.A. Lourakis, Real time
  Tracking of Multiple Skin-Colored Objects with a
  Possibly Moving Camera, in proceedings of the
  European Conference on Computer Vision (ECCV 04),
  Springer-Verlag, vol. 3, pp. 368-379, Prague,
  Chech Republic,May,11-14,2004.
• OpenCV Library Wiki, "http//opencvlibrary.source
  forge.net/".