A Packet Processor for a Learningbased Routing Protocol

1
A Packet Processor for a Learning-based Routing
Protocol

• Taskin Kocak and Hakan Terzioglu
• Dept. of Electrical and Computer Engineering
• University of Central Florida, Orlando, FL 32816
• tkocak_at_cpe.ucf.edu

2
Introduction - RNN

• Random neural networks (RNN) is an analytically
  tractable spiked neural network model that has
  been implemented in software for a wide range of
  applications for over a decade.
• RNN, developed by Gelenbe, has some advantages
  over other models
• Closer to biophysical reality
• Mathematically more tractable
• RNN applications
• Image processing
• Mine detection
• Function approximation
• Magnetic resonance imaging
• Automatic target recognition

3
Random Neural Network Model

• Impulses with unit amplitude propagate through
  the network
• 1 excitation signal
• -1 inhibition signal
• ki(t) instantaneous potential of neuron i
• ?i arrival rate of exogenous excitation signals
• ?I arrival rate of exogenous inhibition signals
• Wji arrival rate of excitation signals from
  neuron j
• W-ji arrival rate of inhibition signals from
  neuron j
• r rate of fire ? Wji W-ji

4
State of the RNN

• Steady state probability of neuron i being
  excited

5
Introduction - CPN

• IP networks must evolve to meet the performance
  demands of todays and tomorrows users.
• Active networks have received a lot of attention
  in the last decade.
• Cognitive Packet Network (CPN) has been proposed
  recently by Gelenbe and shows similarity with
  discrete active networks.
• CPN assigns a QoS Goal to packets
• Goal metric which defines the success of the
  outcome
• i.e. G ?delay ?loss
• Network uses the Goal to provide a best-effort
  attempt at satisfying the QoS requirements
• Employs intelligent based routing to pursue the
  Goal

6
Cognitive Packet Network (CPN)

• Smart Packet (SP)
• probe the network for optimum routes
• next hop is determined using the output of the
  Random Neural Network (RNN) within the current
  router
• Acknowledgement Packet (ACK)
• travels reverse route of SP
• carries network data that is used to calculate a
  Reward value relevant to the Goal of the packet
• Reward value is used to update the status of the
  RNN
• Payload Packet (PP)
• carries payload
• source routed using best path information
  obtained from SP-ACK action

7
Network Processors

• Network processing field has seen an
  unprecedented growth in the last few years due to
  never-ending expansion in the Internet traffic
• Almost all advanced routers have a network
  processor at the heart of their design, which
  allows them to perform at wire-speed
• A network processor is a programmable and
  configurable processor that has been optimized to
  perform networking specific functions, such as
  packet classification, packet modification,
  buffer management, and packet forwarding

8
CPN Network Processor Architecture
9
Smart Packet Processor

• Determines next hop for incoming SPs based upon
  related RNN status
• Updates stored RNN models using data extracted
  from ACK
• Stores RNN models indexed by QoS, Source and
  Destination (QSD)

10
RNN for the CPN

• The function of the RNN in the CPN is to capture
  the effect of unpredictable network parameters
  and convert it into a routing decision
• Each QoS-Source-Destination (QSD) combination
  has its own RNN model
• RNN has fully recurrent topology, 2n2 weights
• Each neuron corresponds to a port of the router
• Neuron with highest output value (q) represents
  next port

11
Reinforcement Learning Algorithm

• Let Reward R 1/Goal (G)
• Compute Threshold T
• Tl aTl-1 (1-a)Rl ,where 0lt a lt1
• If Tl-1 Rl
• --Reward
• w(i, j ) w(i, j ) (Rl - Tl-1),
• w-(i, k) w-(i, k) (Rl - Tl-1)/(n - 1), k ?
  j ,
• Else
• -- Punish
• w(i, k) w(i, k) (Tl-1 - Rl)/(n - 1), k ? j
  ,
• w-(i, j ) w-(i, j ) (Tl-1 - Rl)
• Normalize
• w(i, j ) ? w(i, j ) rOLD/ri

12
RNN Neuron array
13
Neuron Architecture
14
Weight Storage Table Architecture

• Table Controller interacts with SP Interface and
  RL Algorithm
• CAM stores QSD
• size is 16 by 68 bits
• RAM stores weights, thresholds, and port s
• size is 16 by 164 bits

15
Execution of the learning algorithm
16
Payload Packet Switch

• Payload packet switch forwards the payload
  packets to the next hop in the network
• The routing information is stored in the
  cognitive map within the packets header
• Once the next hop is extracted, the packet is
  sent to the output queue of the I/O port

17
Acknowledgment Mailbox

• Acknowledgment packets deposit the relevant data
  measurements into the mailbox
• In our hardware design, the mailbox also
  calculates a reward value from the measurement
  parameters
• This reward value is used to reward or
  punish the routing algorithm with respect to
  the decision made at the current node when
  routing the smart packet

18
System Controller

• System controller handles all the communications
  between the packet processing units and the I/O
  ports
• This multitasking unit is interfaced with other
  modules with 32-bit buses to avoid bottlenecks
  associated with a common bus
• Fetches the packets from the I/O ports in a
  round robin fashion
• If an input port is congested, the priority of
  service will be given to that port

19
I/O port architecture

• Input and output queues can store up to 30
  memory addresses in 12-bit registers.
• Dual Port RAM stores incoming and outgoing
  packet data.
• I/O controller manages the communication between
  the DPR, queues and system controller

20
Simulation

• CPN packet processor is implemented in VHDL
• An example network is created from the instances
  of synthesized processors

21
Circuit implementation details (SPP)
22
Performance evaluation (SPP)

• The CAM component (16 words of 68 bits each) can
  provide
• 362 M searches/s.
• The maximum number of smart packets that can be
  processed
• is 70 M packets/s.

23
Simulation Results
Path 8-1-3-5 rewarded and then continually
selected by flow Disconnection between CPN 3 and
5 at t 15000 ns Smart Packets adapt and find
alternate route
24
Simulation Results
Reconnection between CPN 3 and 5 at t 20000 ns
Smart Packets adapt again
25
Performance measurements Latency of smart and
ack packets
SP header cognitive map
payload AckP header
cognitive map
26
Performance measurements Throughput of smart
packets
27
Performance measurements Loss probability of
smart packets
28
Conclusions and Future Work

• Design and implementation approaches for CPN
  router chip are presented
• Smart packet processor design details are
  discussed
• Component and system level network simulations
  are conducted
• Investigate if the design can be converted to
  instruction-set based one
• Realization of CPN model on commercial NPUs

29
ANCHOR Workshop