3. Applications

1
3. Applications

• Function approximation t ? yt target, y
  actual output
• Neural controlr referenceu control efforty
  system output r ? y

2
Neural Control Schemes

• Supervised control
• Hybrid control
• Model reference control
• Internal model control
• Adaptive control
• Reference C.L. Lin H.W. Su, Intelligent
  control theory in guidance and control system
  design an overview, Proc. Natl. Sci. Counc. ROC
  (A), pp. 15-30.

3
Supervisory Control

• Neural network ? inverse NN controller
• The neural controller in the system is utilized
  as an inverse system model.

4
Hybrid Control

• Generalized learning (off-line learning)A rough
  approximation to the desired control law? Drive
  the plant over the operating range and
  without instability
• Specialized learning (on-line learning)Improve
  the control provided by the NN controller

5
Model Reference Control

• Must define its input-output pair r(t), yR(t)
  in advance
• Attempts to make the plant output y(t) match the
  reference model output asymptotically.
• The error e(t) is used to adjust the weights of
  an neural controller.

6
Internal Model Control

• The NN plant model is first trained off-line to
  emulate the controller plant dynamics directly.
  During on-line operation, the error is used as a
  feedback signal and passed to the NN controller.
• The effect of the NN controller is to subtract
  the effect of the control signal from the plant
  output, i.e., disturbances.
• The IMC plays a role as a feedforward controller
  and can cancel the influence due to unmeasured
  disturbances.

7
Adaptive Control

• The tracking error cost is evaluated according to
  some performance index.
• The result is then used as a basis for adjusting
  the connection weights of the neural networks.
• The weights are adjusted on-line using basic
  backpropagation rather than off-line.

8
Paper Study 1

• Practical Stability Issues in CMAC Neural Network
  Control Systems
• Fu-Chuang Chen Chih-Horng Chang
• IEEE Trans. on Control Systems Technology,
• Vol. 4, No. 1, pp. 86-91, 1996

9
Abstract

• CMAC is a practical tool for improving existing
  nonlinear control systems.
• CMAC can effectively reduce tracking error, but
  can also destabilize a control system which is
  otherwise stable.
• Quantitative studies are presented to search for
  the cause of instability in the CMAC control
  system.

10
I. Introduction

• CMAC is basically a look-up table method, very
  easy to implement, and at the same time it is a
  powerful and practical tool for nonlinear
  control.
• There has been convergence result on the CMAC
  learning.

11
Main Purpose of This Paper

• To introduce the CMAC control system from an
  industrial point of view.
• To describe the unstable phenomenon.
• To quantitatively study how the system parameters
  such as control gain, quantization,
  generalization, learning rate, etc., are related
  to the instability of the system.
• To suggest ways to improve system stability.
• To provide some experience evidence.

12
II. CMAC Control System
Use a workable traditional controller to
stabilize the plant and to help the CMAC learn to
provide precise control.
13
Functioning of CMAC

• Initially the CMAC table is empty.
• In each time step k, the CMAC involves a recall
  and a learning process.

14
Recall Process

• Uses Yd(k1) and Y(k) as the address to generate
  the control signal from CMAC table, where Yd(k1)
  is the desired system output for the next time
  step.
• CMAC has two inputs and one output.

15
Leaning Process

• U(k) is treated as the desired output to modified
  the content stored at location Y(k) and Y(k1),
  where Y(k1) is the actual system output for the
  next time step k1.
• To speed up the initial learning and to achieve
  better generalization, the generalization
  technique is employed, i.e., each input vector to
  CMAC for recall and learning will map to a number
  of memory locations instead of only one memory
  location.

16
Function Approximation

• How precisely the CMAC can approximate a function
  is mainly determined by the quantization in each
  dimension of the input vector.
• Reducing quantization would quickly increase the
  memory demand for storing the CMAC table.

17
Table Update Mechanism

• Gradient-type learning rule
• Wi(k1) Wi(k) ? ? U(k) ? Uc(k) / g
• g the size of generation
• Wi the content of the ith memory location,
  there being q locations to be updated
• ? the learning rule
• U the correct (desired) data
• Uc the current (actual) data

18
III. A Typical Simulation Study

• Proportional gain P 1.4
• Learning rate ? 0.1
• Generalization 50
• Quantization 5/500(meaning five units divided
  into 500 divisions)
• Reference command sin(2?k/200) with each
  sinusoidal cycle consisting of 400 time steps

Tracking error
Number of cycles
19
A Typical Simulation Study

• In the first five cycles, the system is solely
  controlled by the P controller.
• The CMAC is added at the 6th cycle, and then the
  error reduces quickly and significantly.
• The error remains small for some time, and then
  it diverges around the 143th cycle.

Control output
Number of time steps
20
Discussion

• The CMAC can significantly reduce the tracking
  error.
• CMAC can destabilize a control system which is
  otherwise stable. The unstable phenomenon
  certainly comes from the interactions between the
  proportional controller and the CMAC network.

21
Discussion

• The proportional controller can not be removed
  even when the magnitude of the proportional
  control is very small compared with that of the
  CMAC (i.e., when the system output error has been
  significantly reduced). Otherwise, the good
  tracking can not be maintained.

22
Growth of Oscillation
90th cycle
8th cycle
155th cycle
40th cycle
23
V. Method for Improving
System Stability

• The continued learning of CMAC after the tracking
  error has reduced is the major cause of the
  instability.
• Stopping the CMAC learning has two drawbacks.
  First, it can be difficult to determine when to
  stop the CMAC learning.Second, if the CMAC stops
  learning, then the CMAC control system cannot
  respond to any change in the reference command.

24
Modified Learning Rule

• To effectively stop the CMAC learning when the
  tracking error is small, but at the same time
  allow the system to respond to any change in the
  reference command, a deadzone is added to the
  CMAC updating rule.Wi(k1) Wi(k) ? ? D U(k)
  ? Uc(k) / gwhere

25
VI. Experiment
26
Paper Study 2

• Intelligent Controller Using CMACs with
  Self-Organized Structure and Its Application for
  a Process System
• T. Yamamoto, H. Yanagino M. Kaneda
• Proceedings of 1997 IEEE , pp. 76-81

27
Abstract

• This paper describes a design scheme of
  intelligent system consists of some CMACs.
• Each of CMACs is trained for the specified
  command signal.
• A new CMAC is generated for unspecificed command
  signals, and the CMAC whose command is nearest
  for the new command signal, is eliminated.
• The proposed intelligent controller can be
  designed with relatively small memories.

28
1. Introduction

• The CMACs included in the intelligent controller
  are trained in both off-line and on-line learning
  process for each of the specified command
  signals.
• For a unspecified command, a new CMAC is
  generated.
• The initial weights are set by employing the
  linear interpolation to the trained weights
  included in two CMACs whose command signals are
  nearest for the new command signal.
• The CMAC corresponding to the nearest command
  signal is eliminated.
• The proposed intelligent controller can be
  designed with relatively small memories.

29
3. Intelligent Controller Design
controller
30
Outline

• The input signals to each CMAC are the control
  error signal and the difference, that is, the
  two- dimensional CMACs are equipped in the
  intelligent control system.
• By including the command signal as one of input
  signals in the CMAC, the intelligent control
  system can be constructed by using only a
  three-dimensional CMAC.

31
Off-line Learning Process

• w(t) command signalu(t) teacher signal
• Updated ruleh 1, 2, , K total number of
  the selected weightsg1(t) the gradient to
  update the weights

a1, b1, c1 positive cont.
32
On-line Learning Process

• The last weights obtained in the off-line
  learning are used as initial ones in the on-line
  learning.
• Updated rule k the time-delay of the
  systemg2(t) the gradient to update the weights

a2, b2, c2 positive cont.
33
Off-line vs On-line

• In the off-line learning process, the teacher
  signal u(t) is generated by a certain control
  law, e.g., PID control law and human experts.
• u(t) is utilized in order to determine the
  initial weights in the on-line learning of the
  CMAC.
• In the on-line learning process, the teaching
  signal u(t) can not be obtained.
• The desired reference model output ym(t) is
  introduced, and the on-line learning is performed
  so that the system output y(t) approaches to
  ym(t).

34
Self-organized Structure

• A new CMAC is generated for a new command signal
• The initial weights includes in the new CMAC are
  set by employing the linear interpolation to the
  trained weights included two CMACs which are
  nearest for the new command signal
• The CMAC whose command signal is nearest for the
  new one is eliminated.

35
4. Experimental Results

• Air pressure control system
• Control object regulate the air pressure y to
  any desired values by manipulating the control
  value angle u.
• In order to obtain u(t), PID control law is
  employed for this control system.

36
Control Result
Off-line learning(20 iterations)
Conventional PID control
37
Control Result (cont.)
On-line learning(after 5 more iterations)
Unspecified command signal