PowerPoint Presentation - Telerobotics

1
Integration of Internet and real time control
systems in telerobotics
2
Introduction

• Many devices are today accessible through
  Internet
• Web cams
• Washing machines, coffee pots and other
  appliances
• Robots
• The term control is referred to the possibility
  of sending a list of commands to the remote device

3
Introduction

• Note
• The control loop is not closed across the network
• The remote equipment executes the commands under
  the supervision of its own local controller
• Why?
• The unpredictable performance does not allow the
  realization of an Internet-based, real-time
  control system, in which the control loop is
  closed across the network

4
Introduction

• Question
• If the characteristics of the Internet connection
  in use are known, is it possible to implement a
  real-time, closed-loop control system,
  integrating that connection?
• Answer
• JBIT Java Based Interface for Telerobotics a
  telerobotics equipment for real-time
  teleoperation over Internet, with visual and
  force feedback

5
Introduction

• Closing the loop across Internet

6
Summary

• Internet modeling
• Delays, losses and bandwidth
• Control of dynamic systems with variable
  time-delay
• Solutions available in literature
• Packet loss handling
• The JBIT project

7
Internet modeling

• Internet can be considered as a strongly
  connected network of computers, communicating
  with each other using packetswitched protocols

8
Internet modeling

• From source to destination, each packet traverses
  several nodes
• Each node has different throughput, routing
  policy, and buffering and queues management
• Each node handles many other data flows, this
  resulting in random service time

9
Internet modeling

• When the data flow between end users exceeds the
  available bandwidth, congestion occurs
• The available (or marginal) bandwidth for a
  single user depends on the data flow generated by
  other users, sharing the same physical connection

10
Internet modeling

• End-to-end connection modeled by
• Average delay
• Available throughput
• Delay statistics/variations
• Packet loss statistics
• Simple experiments to get the model
• ICMP packet injection and Round Trip Time (RTT)
  measurement

11
Internet modeling

• RTT vs. Time

Short term Long Term (1 week)
12
Internet modeling

• RTT distribution
• Depends on the number of nodes traversed
• 10000 km connection
• 30 km connection

17 hops
5 hops
13
Internet modeling

• RTT distribution

10000 km
30 km
Exponential-like Gaussian-like
14
Internet modeling

• RTT phase plots (RTTn1 vs. RTTn)

rttnk1 rttnk P/? - ?
Non-congested Congested
Note P is the packet length, ? is the channel
throughput, ? is the period of the probing packet
15
Internet modeling

• Available bandwidth can be obtained by injecting
  a data flow that brings the connection close to
  saturation.
• The closer to saturation, the higher the packet
  loss rate

16
Internet modeling - Summary

• Available bandwidth, average delay, delay
  deviation and loss rate can be easily obtained
  with simple injection of probing packets
• Saturation can be avoided by setting the upper
  limit of the application throughput below the
  available BW

17
Control over Internet

• Closing the loop across Internet

18
Control over Internet

• Few solutions available in literature for
  time-varying delay
• All algorithms are designed on the basis of
  maximum delay and its max. derivative
• Structural constraints
• With Internet, short interruptions may occur
• Decentralized control is preferred
• In case of interruptions, it must be allowed to
  have reduced perfomance

19
Control over Internet

• Other solution provide a pair of queues to even
  out the delay jitter
• In this case, all constant-delay techniques can
  be applied (e.g. Smith predictor)

20
Control over Internet

• Which protocol is best suited for Internet-based
  control?
• TCP Acknowledge mechanism interrupts control
  loop
• RTP Oriented to audio-video streaming
• UDP No detection of missing packets
• Enhanced UDP
• Packet loss detection and time stamping

21
Control over Internet

• Other problem random packet losses
• Losses may be single or burst (more than one
  packet lost sequentially)
• Burst losses occur if the data rate is over the
  maximum available BW
• Data flow must be adapted to operating conditions
• Monitoring of available bandwidth through RTT
  on-line measurement

22
Control over Internet

• Single losses can be handled with a predictor
• The prediction is used instead of the missing
  data
• This works in conjunction with the buffering
  mechanism and a packet loss detection procedure
• Based on E-UDP features

23
Control over Internet - Summary

• Not all applications are suitable for IP-based
  control
• The application must survive even in case of
  long interruptions
• Decentralized control must be allowed
• Monitoring of connection characteristics and
  adaptation of the controller to operating
  conditions
• Data recovery must be provided

24
Case study Telerobotics
Bi-directional data exchange
25
Case study Telerobotics

• This application is well suited for IP-based
  control
• Decentralized structure
• Several control algorithms for constant time
  delay are available

s
26
Telerobotics at the Industrial Electronics Lab.

• Research started in 1995
• Biorobotics Lab. UW Seattle
• Telerobotics Group Jet Propulsion Laboratory
• LVR - Università di Orleans-Bourges
• Design and realization of an Internet-based
  telerobotic equipment with force feedback
• Enanched-UDP
• JBIT
• P_at_dus - OTELO

27
Web-based telerobotic systems

• Several systems are available
• None allow real time feedback and control
• CGI interface and script-like commands
• Connection is open when the query is sent and
  closed upon reply arrival
• no continuous video streaming
• e-mail or updated environment picture as feedback

28
JBIT Java-Based Interface for Telerobotics

• Targets
• to enable any Internet user to access a remote
  robot, with real time video, VR and (possibly)
  force feedback
• low cost (overall!)
• no special tools (SW HW)

29
JBIT solutions - 1

• Cost
• Freeware
• Low cost camera (Quickcam)
• Commercial force display (Microsoft Side Winder
  Pro) as master
• Direct-drive, low cost robot as slave
  (cannibalized from HDDs)
• Sensorless force feedback

30
JBIT solutions - 2

• Real-time visual and force feedback
• Java servlets for fast server response
• compressed video streaming (H.263) for video
• VR-based interface to improve the visual feedback
  performance
• coordinating force feedback queuing data
  recovery to cope with IP delays and losses

31
Structure of JBIT system
32
JBIT Client-server structure
33
JBIT Client-server structure

• Server (implemented with Java servlets)
• Waiting for client requests
• Access control, timeout
• Data exchange between client and robot (no direct
  access form client to robot)
• Video encoding (H.263) and streaming
• Monitoring of available bandwidth
• Adaptation of video and control output rate

34
JBIT Client-server structure

• Client (implements user interface)
• Communication with servlets
• Selection of operating mode
• Radio button and active joystick interface
• VR interface
• Video decoding and playing (Java Media Framework
  - JMF)

35
Client interface
36
JBIT - Force feedback

• Up to 200 Hz sampling rate on a LAN, 50 Hz with a
  14.4 kbps modem

37
JBIT - Force feedback

• Perception of the remote environment
• Depends on the connection delay

Remote tracking of a square object
38
Conclusions

• The realization of an IP-based closed loop
  control system is not a trivial task.
• Precise knowledge of the connections
  characteristics is needed
• Such characteristics must be constantly monitored
• Reconstruction of lost data should be ensured

39
Conclusions

• Need for some smart strategies to deal with long
  term interruptions
• Few applications are suitable for IP-based
  control
• Several issues still to be addressed in order to
  achieve a general solution for IP-based control

40
Conclusions

• Some help may arrive from new network
  technologies
• Hard real-time connections (e.g. Tenet)
• New version of Internet (IPv6)
• RSVP (Resource Reservation Protocol)
• New video compression (MPEG4)
• It will be possible to have connections with
  prescribed Quality of Service (QoS), in terms of
  throughput, losses and delays
• Reliable control of remote equipment using
  Internet could be implemented more easily.

41
On-going project P_at_dus
PATIENT STATION (SLAVE)
ECHO PROBE
PC
ECHOGRAPH
SLAVE
PATIENT
EXPERT STATION (MASTER)
COMMUNICATIONCHANNEL
PC
HAPTIC MASTER
EXPERT
42
Tele-echographies