Acquisition of Control Knowledge of Nonholonomic System by Active Learning method

1
Acquisition of Control Knowledge ofNonholonomic
System by Active Learning method
?????????????????????????

• Yoshitaka Sakurai Nakaji Honda Junji
  Nishino
• Presented by Pujan Ziaie

2
Paper Information

• Journal of Advanced Computational Intelligence
  Intelligent Informatics
• Received August 28,2002 accepted December
  13,2002
• Proc. of 2003 IEEE International Conference on
  Systems
• pp.2400--2405 (2003.10)

3
About author

• Yoshitaka Sakurai (P.H.D. Student)

• University of Electro-Communications
• Department of systems Engineering
• Honda Lab.

4
Introduction

• ALM (Active Learning Method)
• IDS (Ink Drop Spread)
• Simulation for Gymnastic Bar Action
• Mathematical Model Equations
• Active Learning Approach
• Conclusion

5
Active Learning Method

• Why ALM?
• No need to now the System Inner Structure
• Improving performance by its own
• Characteristics
• Construction
• Modeling

6
ALM Characteristics

• Using SiSO systems
• Choosing most effective data
• Accumulation of knowledge by Experience
• Reinforcement Learning (reward or punishment)
• Estimation of overall information By fragmentary
  information

7
ALM Construction

• Similar to human learning

Knowledge Acquisition Part
Controller
Trial Error
IDS
Storage Of I/O data
Sampling Rule
Modeling
Database
Control Rule
Data collection
Evaluation
System Under Control
8
ALM Modeling (1)

• Dividing MIMO System to SISO Systems
• Dividing input Domains to fuzzy regions
• Extracting the continues narrow path
• Calculating the output by Sum of the
  (Adaptability of each region region-output)

Combination Rule
MIMO System
SISO
SISO
Combination Rule
SISO
9
ALM Modeling (2)

• Example 2-Input gt 1-Output

yM
yL
y
ßL
ZL
ßM
ZM
X2
X2
b
VS S M L VL
b
y ßvs Zvs ßs Zs ßM ZM ßL ZL
ßVL ZvL
X1a X2b
X1
y ßM ZM ßL ZL
a
10
Ink Drop Spread method

• What is IDS?
• Extract narrow path by using fuzzy process on
  input-output data
• Why using IDS?
• Create a continuous narrow path
• Measure the data distribution amount (extracting
  the most effective input)

11
IDS Algorithm (1)

• Using irradiation pyramid on data plane

12
IDS Algorithm (2)
Data plan
Projected plan
Narrow path
Combining the lights
13
IDS Algorithm (3)

• Sample of IDS

Gathering more Data ( through feedback )
Gathering more Data
14
Control process (1)

• Defining the control structure
• Dividing inputs into regions according to their
  range
• Selecting most efficient input for the required
  output ( by human or controller)
• Defining evaluation rule for selecting suitable
  data

15
Control process (2)

• Control cycle
• Gathering data by using control rules
• First time using random numbers
• After first time, using the developed controller
• Evaluate the gathered data
• Improve the partial knowledge function (in case
  of proper data)
• Repeat from step 1

16
Control process (3)

• Output calculation method
• Remove the most efficient input from inputs
• Build input states tree according to valid fuzzy
  regions
• Extract a narrow path of the most efficient
  input and output for each leaf of the tree
• Calculate the final output value by sum of output
  of each node multiplied by the adaptability of
  that node.

From narrow path
By multiplying the Membership values Of nodes
from root To the leaf
17
Gymnastic Bar Action

• Model of Bar Gymnast
• 4 joints 5 links
• Link 0 is not driven
• ?0 is dependent of the position of center of
  gravity of the model and shape of posture.
• The mass of the head is assumed to be 0.

• GOAL achieve the largest swing angel

18
Equations

• ?i relative angle between link i-1 and link i at
  each joint i. (i0..4)
• T kinetic energy
• V potential Energy (gravity)
• L T-V gt Lagrangian equation

• Ii moment of inertia
• xi, yi coordinates of center of gravity of the
  ith link
• Ni torque applied on each joint i

19
Acquisition of knowledge

• Does a little Kid learn the gymnastic Bar, by
  Solving lagrangian equation?!

NO!

• Trying to Learn from the environment by trial
  and error

20
ALM against Model of Bar gymnast
IDS Diagrams
Knowledge Acquisition Part
Controller
Probability based on distribution
IO Model
Sampling Rule
IDS
Modeling
Control Rule
Database
Evaluation
Data collection
Simulator
Sequential Database
After some specified time
Comparing with last most Swing angles
21
Simulation properties

• Sampling rate each 1/1000 Sec
• Evaluation each 2 minutes
• Angle range division
• ?0 -180 to 180 gt 8 MFs
• ?1 0 to 130 gt 5 MFs
• ?2 -180 to 0 gt 5 MFs
• ?3 -130 to 30 gt 5 MFs
• ?4 0 to 130 gt 5 MFs
• Most Effective input of each output (joint
  Torque) gt the angle of the same joint

22
Simulation result (1)
23
Simulation result (2)
24
Conclusion

• ALM is a Strong flexible method against some
  complicate control problems
• Mathematics is completely useless for many
  control problems
• Advantages of this approach
• flexibility
• easiness
• disadvantages
• imperfect information collecting rule
• still too crisp

25
Why did I choose this paper?

• I liked it.
• It was quite a challenge
• It was brand new
• I have some ideas to improve it
• using fuzzy approaches for output
• correcting membership functions instead of adding
  new data

26
acknowledgment

• Special thanks to
• Sakurai-san for giving me his time and answering
  my question
• Yamazaki-san who helped me to write the Japanese
  translation of technical words
• Serata-san who set me an appointment with
  Sakurai-san

27
Thank you all for listening

• Any easy questions?!

28
Sampling rule

• Probability based on distribution function

probability function
y
y
Xn
29
input tree

• i.e.
• y four inputs (x0..x3)
• x1 is the most efficient

x0
ßob
ßoa
x2
x2
ß2c
y ß_L1_S1f_L1_S1(x1) ß_L1_S2f_L1_S2(x1) ß_L1
_S3f_L1_S3(x1) ß_L1_S4f_L1_S4(x1)
ß2c
ß2d
ß2d
x3
x3
x3
ß3e
x3
ß3f
adaptability of this state(1) ß_L1_S1 ß0b
ß2c ß3e
y
using partial knowledge function
x1 output for this state(1) f_L1_S1(x1)