Clustering%20Sensor%20Data%20for%20Terrain%20Identification%20using%20a%20Windowless%20Algorithm

1
Clustering Sensor Data for Terrain Identification
using a Windowless Algorithm

• Philippe Giguère
• Gregory Dudek

2
Outline

• Problem
• Proposed algorithm
• Evaluation on synthetic datasets
• Application to terrain identification
• Conclusion, future work

3
Exploring the World
Richard Altendorfer, Ned Moore, Haldun Komsuoglu,
Martin Buehler, H. Benjamin Brown Jr., Dave
McMordie, Uluc Saranli, Robert Full, Daniel E.
Koditschek RHex A Biologically Inspired Hexapod
Runner. Auton. Robots (AROBOTS) (2001)
4
Autonomous Terrain Identification
5
UnsupervisedLearning of Terrains
6
UnsupervisedLearning of Terrains
7
Assumptions

• Representative features
• Time continuity

8
Time Series Clustering

• Time-Window
• J. Kohlmorgen and S. Lemm. An on-line method for
  segmentation and identification of non-stationary
  time series. In IEEE Neural Networks for Signal
  Processing Workshop 2001.
• S. Lenser and M. Veloso. Automatic detection and
  response to environmental change. In Proceedings
  of the ICRA, May 2003.
• Regularization (smoothness)
• J. Kohlmorgen, S. Lemm, G. Rätsch, and K.-R.
  Müller. Analysis of non-stationary time series by
  mixtures of self-organizing predictors. In IEEE
  Neural Networks for Signal Processing Workshop,
  2000.

9
Potential Issues

• Time-windows
• Transitions
• Complex-shaped distributions
• Regularization
• Right regularization constant l?

10
Classifier Definition
11
Classifier Training
12
Classifier Training
13
Proposed Algorithm

• A classifier generalizing
  over local distances in feature space
• e.g., k-Nearest Neighbors
• Training rule

14
Optimization Simulated Annealing
Set q rand while (temp gt finaltemp) qnew
RandomChange(q) if E(qnew,X) lt E(q,X) q
qnew Always accept good elseif
P(E(q,X),E(qnew,X),temp) gt rand q qnew
Sometimes accept bad end decrease temp
Cooling end return q
15
Testing on Synthetic Datasets

• Classifiers
• Linear Separator
• Mixture of Gaussians
• K-Nearest Neighbors
• Distributions

16
Synthetic Datasets Generation
17
Synthetic Datasets Generation
18
Linear Separator
Segment Length 3
19
Synthetic Datasets
Number of classes in (bracket)
20
k-Nearest Neighbors
k 30
Symbol Cluster Assignment
Segment Length 5
21
Played-back through aSliding Window
22
Comparison withWindow-based Clustering
Results
S. Lenser and M. Veloso. Automatic detection and
response to environmental change. In Proceedings
of the ICRA, May 2003.
23
AQUA Robot (Based on RHex)
G. Dudek, M. Jenkin, C. Prahacs, A. Hogue, J.
Sattar, P. Giguere, A. German, H. Liu, S.
Saunderson, A. Ripsman, S. Simhon, L. A.
Torres-Mendez, E. Milios, P. Zhang, I. Rekleitis.
A Visually Guided Swimming Robot, Proceedings of
the 2005 IEEE/RSJ International Conference on
Intelligent Robots and Systems, pp. 1749-1754,
2005.
24
AQUA Sensors
25
Features Dimensionality Reduction
26
Features Dimensionality Reduction
27
Features Dimensionality Reduction
28
Features Dimensionality Reduction
29
Terrains

• Robot manually driven over
• Grass
• Tilled Earth
• Snow
• Ice
• Linoleum

30
2 Terrains (Linear Separator)
31
5 Terrains Data
32
5 Terrains Data
33
5 Terrains (Gaussian Mix. Model)
Classification Rate 91
34
5 Terrains (Gaussian Mix. Model)
35
Conclusion

• Clustering combining time and feature
• No
• Time window
• Regularization
• Training a classifier with cost function
• Unsupervised learning much easier
• Experimentally validated on hexapod robot

36
Future Work

• More terrains
• Accelerated optimization
• On-line version
• Finding number of clusters
• Other problems
• Texture segmentation

37
Questions?
38
Comparing with HMM
39
Dimensionality Reduction
40
1st Part Finding Projection
41
Cost Function Formulation
42
k-NN Few Points
Segment Length 3
43
k-NN 3-to-1 Ratio
Segment Length 3
44
k-NN Overlapping
Segment Length 3
45
k-NN 6 Classes
Segment Length 3
46
Clustering Sensor Data for Terrain Identification
using a Windowless Algorithm

• Philippe Giguère
• Gregory Dudek

47
Outline

• Problem
• Proposed algorithm
• Evaluation on synthetic datasets
• Application to terrain identification
• Conclusion, future work

48
Exploring the World
Richard Altendorfer, Ned Moore, Haldun Komsuoglu,
Martin Buehler, H. Benjamin Brown Jr., Dave
McMordie, Uluc Saranli, Robert Full, Daniel E.
Koditschek RHex A Biologically Inspired Hexapod
Runner. Auton. Robots (AROBOTS) (2001)
49
Autonomous Terrain Identification
50
UnsupervisedLearning of Terrains
51
UnsupervisedLearning of Terrains
52
Assumptions

• Representative features
• Time continuity

53
Time Series Clustering

• Time-Window
• J. Kohlmorgen and S. Lemm. An on-line method for
  segmentation and identification of non-stationary
  time series. In IEEE Neural Networks for Signal
  Processing Workshop 2001.
• S. Lenser and M. Veloso. Automatic detection and
  response to environmental change. In Proceedings
  of the ICRA, May 2003.
• Regularization (smoothness)
• J. Kohlmorgen, S. Lemm, G. Rätsch, and K.-R.
  Müller. Analysis of non-stationary time series by
  mixtures of self-organizing predictors. In IEEE
  Neural Networks for Signal Processing Workshop,
  2000.

54
Potential Issues

• Time-windows
• Transitions
• Complex-shaped distributions
• Regularization
• Right regularization constant l?

55
Classifier Definition
56
Classifier Training
57
Classifier Training
58
Proposed Algorithm
59
Optimization Simulated Annealing
60
Testing on Synthetic Datasets

• Classifiers
• Linear Separator
• Mixture of Gaussians
• K-Nearest Neighbors
• Distributions

61
Synthetic Datasets Generation
62
Synthetic Datasets Generation
63
Linear Separator
Segment Length 3
64
Synthetic Datasets
Number of classes in (bracket)
65
k-Nearest Neighbors
k 30
Segment Length 5
Symbol Cluster Assignment
66
Played-back through aSliding Window
67
Comparison withWindow-based Clustering
Results
S. Lenser and M. Veloso. Automatic detection and
response to environmental change. In Proceedings
of the ICRA, May 2003.
68
AQUA Robot (Based on RHex)
G. Dudek, M. Jenkin, C. Prahacs, A. Hogue, J.
Sattar, P. Giguere, A. German, H. Liu, S.
Saunderson, A. Ripsman, S. Simhon, L. A.
Torres-Mendez, E. Milios, P. Zhang, I. Rekleitis.
A Visually Guided Swimming Robot, Proceedings of
the 2005 IEEE/RSJ International Conference on
Intelligent Robots and Systems, pp. 1749-1754,
2005.
69
AQUA Sensors
70
Features Dimensionality Reduction
71
Features Dimensionality Reduction
72
Features Dimensionality Reduction
73
Features Dimensionality Reduction
74
Terrains

• Robot manually driven over
• Grass
• Tilled Earth
• Snow
• Ice
• Linoleum

75
2 Terrains (Linear Separator)
76
5 Terrains Data
77
5 Terrains Data
78
5 Terrains (Gaussian Mix. Model)
Classification Rate 91
79
5 Terrains (Gaussian Mix. Model)
80
Conclusion

• Clustering combining time and feature
• No
• Time window
• Regularization
• Training a classifier with cost function
• Unsupervised learning much easier
• Experimentally validated on hexapod robot

81
Future Work

• More terrains
• Accelerated optimization
• On-line version
• Finding number of clusters
• Other problems
• Texture segmentation

82
Questions?