Learning Automata

1
Learning Automata

• Learns the unknown nature of an environment
• Variable structure stochastic learning automaton
  is a quintuple j,a,b,A,G where
• j(n), state of automaton jj1,,js
• a(n), output of automaton aa1,,ar
• b(n), input to automaton bb1,,bm
• A, is the learning algorithm
• G., is the output function a(n)Gj(n)
• n indicates the iteration number.

2
Learning Automaton Schematic


Action

Environment
Response
b(n)
a(n)
Automaton ltf,a,b,A,Ggt ltstates, actions, input,
learning algorithm, output functiongt
Input
Output
3
Probability Vector

• pj(n), action probability the probability that
  automaton is in state j at iteration n.
• Reinforcement scheme
• If a(n) ai and for j ltgt i (j1 to r)
• pj(n1) pj(n) - g pj(n) when b(n)
  0.
• pj(n1) pj(n) h pj(n) when b(n) 1.
• In order to preserve probability measure,
• S pj (n) 1, for j 1 to r.

4
contd ...

• If a(n) ai
• 
  r
• pi(n1) pi(n) S g(pj(n)) when
  b(n) 0
• j1, jltgti
• r
• pi(n1) pi(n) - S h(pj(n)) when
  b(n) 1
• j1, jltgti
• g(.) is the reward function
• h(.) is the penalty function

5
Schematic of Proposed Automata Model for
Mapping/Scheduling


Machine
Eresp

HC System Model
bsi(n)
asi(n)
Automaton for task si ltasi,bsi,Asigt ltmachines,
environment response, learning algorithm gt
Input
Output
6
Model Construction

• Every task si associated with an S-model
  automaton (VSSA).
• VSSA represented as asi, bsi,Asi, since r s
• asi is set of actions asi m0, m1, , mM-1
• bsi is input to the automaton, bsi 0, 1
• closer to 0 action favorable to system
• closer to 1 action unfavorable to system
• Asi is reinforcement scheme
• pij(n) - action probability vector
• probability of assigning task si to machine mj

7
contd

• Automata model for Mapping/Scheduling
• S-model VSSA is used
• Each automaton is represented as a tuple
  asi,bsi,Asi
• asi m0, m1, , mM-1
• bsi 0, 1
• (closer to 0 - favorable, 1 - unfavorable)
• If ck(n) is better than ck(n-1)
• Ekresp 0 else Ekresp 1
• Translating Ekresp to bsi (n) requires two steps

8
Translating Ekresp to bsi
E1resp
E2resp
Ekresp
mS-11
m0k
m1k
m01
mS-1k
m11
Task s0
Task s1
Task sS-1
bs0
bs1
bss-1
9
contd

• Step 1 Translate Ekresp to msik (n), where
• msik (n) - input to automaton si with respect to
  cost metric ck
• achieved by the heuristics
• Step 2 Achieved be means of Lagrange's
  multiplier
• C
  
  C
• bsi (n) S lk msik (n), i1 to S-1 S lk
  1, lk gt 0
• j1
  j1
• where lk is the weight of metric ck

10
(No Transcript)
11
Schematic of Proposed Automata Model for
Architecture Trades


Machine
Peval

System Solution
bsi(n)
asi(n)
Automaton for component si ltasi,bsi,Asigt ltcomponen
ts, performance evaluation, learning algorithm gt
Input
Output
12
Model Construction

• Every component of the HW system si associated
  with a P-model automaton (VSSA).
• VSSA represented as asi, bsi,Asi, since r s
• asi is set of component types asi c0, c1,
  , cM-1
• bsi is input to the automaton, bsi 0, 1
• 0 performance favorable to system 1
  unfavorable to system
• Asi is reinforcement scheme
• pij(n) - action probability vector
• probability of choosing component si from
  component type cj

13
contd

• Automata model for Architecture Trades
• P-model VSSA is used
• Each automaton is represented as a tuple
  asi,bsi,Asi
• asi c0, c1, , cM-1
• bsi 0, 1
• (0 - favorable, 1 - unfavorable)
• If ck(n) is better than ck(n-1)
• Peval 0 else Peval 1

14
Conclusions

• Adaptive Framework for Mapping and Architecture
  trades
• Automata models allow optimization of multiple
  criteria
• Efficient / gracefully degradable solutions
• Framework construction suitable for tool
  integration
• Mapping algorithm integrated with SAGETM
• Provides a basis for systems design from
  application to the embedded HW