3D Haptics and Robotics

1
3D Haptics and Robotics
Krasimir Kolarov Interval Research
Corporation
HCC Seminar, UC Berkeley, 12/3/98.
2
What is this talk about?

• Introduction to Haptics and Haptic Interfaces
• Commercial and University Haptics
• Force Feedback Devices
• 3D Haptics at Interval Research Corp.

3
Haptic Interfaces

• Haptic - an information processing perceptual
  system that uses inputs from the receptors
  embedded in the skin, as well as in muscles,
  tendons and joints (Loomis and Lederman, 1986)
• hap.tic (haptik) adj. of or having to do with
  the sense of touch tactile (Websters New World
  Dictionary)
• haptic interfaces - devices that measure the
  motion of, and stimulate the sensory capabilities
  within, our hands (as used in human interface
  technology )

4
Unique Characteristics of Haptics

• Haptics relies on action to stimulate perception.
• The haptic system can sense and act on the
  environment while vision and audition have purely
  sensory nature.
• Being able to touch, feel, and manipulate objects
  in an environment, in addition to seeing (and
  hearing) them, provides a sense of immersion in
  the environment that is otherwise not possible
  (Srinivasan, 1995)

5
Other Topics

• Actuators (electrical, hydraulic, pneumatic)
• Sensors
• Tactile Feedback Interfaces (sensors, texture,
  slip, surface temperature)
• Control of Haptic Interfaces (distributed
  computation, quality)
• Physical Modeling (collision detection, surface
  deformation, mechanical compliance, smoothness,
  physical construction)

6
Applications

• Enhancement of GUIs (graphical user interfaces)
  - enable users to feel where the buttons on their
  programs are.
• Computer Games - engaging touch interactions,
  cost-sensitive market.
• Simulation for training humans - to perform tasks
  that require sensorimotor skills (surgery,
  training for naval personnel).
• Interaction with computer-generated 3D data -
  users of CAD/CAM, data visualization and other
  engineering and scientific applications.
• Medicine, Entertainment, Telerobotics.

7
Human Haptics System

• Tactile Sensory System - distinguish vibrations
  up to 1 KHz detection threshold on smooth glass
  plate - 2 m high single dot, 0.06 m high grating
• Kinesthetic Sensory System - bandwidth 20-30 Hz
  JND (just noticeable distance) - 2.5o for finger
  joint, 2o for wrist and elbow, 0.8o for the
  shoulder
• Motor System - human bandwidth for limb motions
  is between 1-10 Hz as a function of the mode of
  operation
• Active Touch with all three systems - stiffness gt
  25N/mm is needed for an object to be perceived as
  rigid

8
Functions of Haptic Interfaces

• to measure the position and contact forces (and
  time derivatives) of the users hand (or other
  body parts).
• to display contact forces and positions (or their
  spatial and temporal distributions) to the user.
• Alerting function - vibrations.
• Premise The sense of touching simple shapes
  could be evoked by programming computers to
  control electromechanical master devices. We can
  build devices that give us a sense of feel when
  controlling remote actions with a high degree of
  dexterity.

9
Categories of Haptics Interfaces

• 1. Free motion, in which no physical contact is
  made with objects in the environment
• 2. Contact involving unbalanced resultant forces
  (like pressing an object with a finger pad)
• 3. Contact involving self-equilibrating forces
  (like squeezing an object in a pinch grasp)
• Additional consideration - we can touch, feel and
  manipulate the objects directly or with a tool

10
Currently Available Haptic Interfaces

• Ground-Based Devices
• joysticks/ hand controllers
• Body-Based Devices
• exoskeletal devices
• flexible (gloves and suits worn by users)
• rigid links (jointed linkages affixed to users)
• Tactile Displays
• shape changers
• shape memory actuators
• pneumatic actuators
• microelectromechanical actuators
• vibrotactile
• electrotactile

11
History of Haptic Interfaces
12
Haptics Research in Universities

• MIT AI Lab
• Salisbury (design of high performance mechanisms
  and sensors)
• MIT Human-Machine Systems Lab
• Srinivasan (understand human haptics, enhance
  human-machine interaction)
• Sheridan (tactile and auditory substitution of
  force feedback for teleoperation)
• MIT Media Lab
• Margaret Minsky (tactile feedback from a graphics
  simulation, home haptics)
• Plesniak (haptics and holographic systems)
• Harvard University
• Rob Howe (tactile display of shape and
  vibrations)
• UNC
• Taylor, Fred Brooks (nanomanipulator force
  feedback, medical research)
• CMU
• Baraff, Vedula (force feedback in interactive
  dynamic simulation)

13
University Research (cont.)

• Stanford University
• ME Dept., Cutkosky (force feedback grasping,
  multi-finger manipulation)
• CS Dept., Khatib, Ruspini (haptics library, force
  control, dynamics)
• CCRMA, OModrian (grand piano simulation, haptics
  for the blind)
• UC Berkeley
• Canny (optimum stability of grasp, dynamic
  simulations)
• Northwestern University
• Ed Colgate (dynamically effects like mechanical
  impedance)
• Japan
• Iwata (6 dof stewart platform joystick Haptic
  Master, mechanical design)
• University of New Mexico, University of Virginia,
  University of Colorado, Rutgers University,
  Georgia Tech, McGill University, Naval
  Postgraduate School, University of Washington,
  Simon Fraser University, ...

14
Industrial Research and Development

• SensAble Devices (PhanTom)
• Immersion Corp. (Impulse Engines, Joysticks)
• Cybernet Systems Corp. (CyberImpact Joystick,
  Steering Wheel, Flight Yoke)
• Microsoft (formerly - EXOS Inc. Power Stick,
  Surgical Simulator, SAFiRE)
• Boston Dynamics (Tangible Reality, Interactive
  Humans)
• VTT, Finland (virtual prototyping)
• Interval Research Corp., MERL, GE Corporate RD,
  High Techsplanations Inc., Army, Navy, ...

15
Immersion Corporation
16
Cybernet Systems Corp.
17
EXOS Inc.
18
SensAble Technologies Inc.
19
Video on Applications (SensAble)

• before that
• bw slides from Web Page
• color slides from Hassers report
• after that color slides from VTT

20
Research Interests
Develop a cooperative graphic and haptic
interface that allows to manipulate and sculpture
3D objects more effectively
21
Goals and Assumptions

• Provide a high level interface to haptic devices
  that
• Complements existing interactive graphic systems
• Works robustly in multi-surface environments
• Provide common framework to allow stable and safe
  haptic control.
• Ability to perform tasks that are not possible
  with the current technology
• The combined graphical and haptic interface to
  3D objects will allow us much richer and powerful
  interaction.

22
Existing Graphic Systems

• Capable of displaying a large number of simple
  polygons at interactive rates (gt20,000 polygons
  at 30Hz)
• Intersecting polygons gaps common
• Topology seldom available (Polygon Soup)
• Gourand/Phong Shading Texture

23
Questions

• How can you support a powerful and general set
  of modeling 3D primitives.
• Allow the haptic server to operate with a great
  amount of autonomy from the host computer and
  simulate a wide range of virtual environments
• Explore issues like latency in manipulating large
  3D data sets

24
Basis for Research

• Test distance/collision calculation and dynamic
  simulation.
• Building a library to support arbitrary complex
  rigid objects.
• Allow the developer to specify constraints
  between the objects in the environment and
  control the motion of objects in the virtual
  world.
• Model the contact forces caused by contact and
  collisions between the objects in the environment

25
Stanford Students Projects

• The Virtual Xylophone.
• The Haptic Roaches.
• Haptic Exploration of rigid 3D objects.

26
Video on Roaches

• before that - color slides on xylophone, roaches
• after that
• color slide on staircase
• slides from the HL talk
• goals
• bounding sphere covering
• bounding sphere hierarchy
• simulating smooth surfaces
• results

27
Issues

• How to display (and compute) elemental sensations
  such as impact, friction, softness, motion and
  constraint.
• How to involve more complicated interactions (the
  Phantom concentrates on forces at the fingertip
  or tool tip). Those include pressure
  distribution, temperature and high-frequency
  vibration.

28
Solutions

• Penalty Based Haptic Systems
• Bounding Sphere Hierarchy
• Virtual Proxy Model
• Surface Properties
• Force Shading/Texturing
• HL Library with Application Programmers Interface
  similar to GL

29
Bounding Sphere Hierarchy
30
The Virtual Proxy
31
Virtual Proxy Description

• A representative object that is constrained by
  obstacles in the environment
• Proxy is reduced to a point (C-space). User
  definable size of proxy.
• Constraint planes locally describe the range of
  potential proxy motion.
• Proxy moves to locally minimize the distance to
  users position
• Haptic device physically moves user to proxys
  position

32
Force Shading
33
Haptic System Implementation

• HL library, syntax similar to GL
• graphic Client/Haptic Server Model
• Bounding Sphere Hierarchy
• - O(log n) growth
• gt 24,000 polygonal primitives on 200 MHz Pentium
  - Stiffness 1800 Newtons/meter
• - gt 1000 Hz servo rate

34
Demonstrations
35
Sample References

• Force and Touch Feedback for Virtual Reality,
  Grigore Burdea, Rutgers Univ., 1996, John Wiley
  Sons.
• The PHANToM Users Group Workshop Proceeding,
  MIT September 1996, 1997, 1998 (published as MIT
  AI Lab Tech Reports).
• Haptics Home Page at Northwestern Univ.
  http//haptic.mech.nwu.edu/
• Ruspini, D., Kolarov, K. and Khatib, O. "The
  Haptic Display of Complex Graphical
  Environments", Computer Graphics Proceedings,
  Annual Conference Series, SIGGRAPH'97, Los
  Angeles, California, September 1997