Introduction to Haptic Rendering

1
Introduction to Haptic Rendering

• Ming C. Lin
• lin_at_cs.unc.edu
• http//gamma.cs.unc.edu/interactive

2
What Is Haptic Rendering?
Master-Slave Systems
Human-in-the-Loop
Robot
Force Feedback
Haptic Device
Human
Simulation
Tactile Feedback
Virtual Reality
Pictures http//haptic.mech.nwu.edu/intro/gallery
/
3
Why Haptics?

• Natural 3D interaction with a simulated
  environment. Not constrained by a 2D projection.
• Additional sensorial input. Visual is dominant.
  Exploit extra dimensions, cheat the user through
  the eyes In case of inconsistencies, visual
  wins.

4
Inter-disciplinary Research
Computer Science
Electrical Engineering
Mechanical Engineering
Haptic Rendering
Control and actuators
Mechanical design
Computation of the forces output by the device
5
To be covered

• Haptic rendering for pen-type devices

Point-Object interaction Object-Object interaction

• 3DOF Haptics(3D force output)
• 6DOF Haptics (3D force torque output)

DOF Degree of Freedom
6
Some History

• Argonne 54, first master-slave systems.
  Masterslave.
• Salisbury 80, independent master and slave.
  Cartesian control of robot arm. Later,
  interaction with computer simulated slave (VR).
• GROPE project UNC 67-90, molecular docking.
• Minsky 90, the Sandpaper.
• Massie Salisbury 94, the Phantom.
• Early 90s to 97, 3DOF haptics.
• Late 90s to today, 6DOF haptics.

7
Control of Haptic Devices

• Impedance devices

• Admittance devices

User
Simulation
User
Simulation
F
x, v
Actuators
Actuators
x
x
F
F

• Collision detection contact response model
• More popular

• Collision detection position constraints
• Bulkier and more expensive
• Very good for virtual walls

8
Control of Haptic Devices

• Impedance Devices

• Admittance devices

9
1KHz Performance Requirement

• The user becomes part of the simulation loop.
• 1KHz is necessary so that the whole system
  doesnt suffer from disturbing oscillations.
• Think of the analogy with numerical integration
  of a system with spring, mass and damper, where
  the frequency of the haptic loop sets the
  integration step.
• The Phantom haptic devices run their control loop
  at 1KHz.
• Consequence we are very limited on the amount of
  computation that we can do.

10
3DOF Haptics Intro

• Output 3D force ? 3DOF haptics
• Limited to applications where point-object
  interaction is enough.
• Haptic visualization of data
• Painting and sculpting
• Some medical applications

11
3DOF Haptics Basic approach

• Check if point penetrates an object.
• Find closest point on the surface.
• Penalty-based force.

x
F
12
3DOF Haptics The problems

• Force discontinuities when crossing boundaries of
  internal Voronoi cells.

Unexpected force discontinuities (both in
magnitude and direction) are very disturbing!
F2
F1
13
3DOF Haptics The problems

• Pop-through thin objects.

After the mid line is crossed, the force helps
popping through.
motion
14
3DOF Haptics Point-to-plane

• Mark et al., 1996.
• For distributed applications. The simulator sends
  the equation of a plane to the haptic loop. The
  update of the plane is asynchronous.
• Forces are computed between the haptic point and
  the plane.
• Possible stiffness is limited, because too stiff
  would bring a jerky behavior at low update rates.

15
3DOF Haptics God-object

• Zilles and Salisbury, 1995.
• Use the position of the haptic interface point
  (HIP) and a set of local constraint surfaces to
  compute the position of god-object (GO).
• Constraint surfaces defined using heuristics.
• Compute GO as the point that minimizes the
  distance from HIP to the constraint surfaces.
  Lagrange multipliers.

16
3DOF Haptics God-object

• Constraint surfaces
• Surfaces impeding motion
• GO is outside (orientation test) and in the
  extension of the surface.
• The HIP is inside the surface.

17
3DOF Haptics God-object

• Constraint plane equations
• Energy function that will account for the
  distance.
• Define cost function using Lagrange multipliers.
• Minimize, solving for x, y, z, l1, l2 and l3.

3 planes at most
18
3DOF Haptics God-object

• Partial derivatives on x, y, z, l1, l2 and l3
  yield 6 linear equations
• In case of less than 3 planes, the problem has a
  lower dimension.

19
3DOF Haptics Virtual proxy

• Ruspini et al., 1997.
• Based on god-object.
• Virtual proxy is a small sphere, instead of a
  point. Use configuration-space obstacles
  (C-obstacles), from robotics.
• More formal definition of constraint planes.
• Implementation of additional features, based on
  relocation of the virtual proxy.

20
3DOF Haptics Virtual proxy

• C-obstacles for a spherical object, is reduced
  to computing offset surfaces at a distance equal
  to the radius of the sphere.
• Check the HIP against the offset surface.
• This is done to avoid problems with small gaps in
  the mesh.

21
3DOF Haptics Virtual proxy

• Finding the virtual proxy is based on an
  iterative search.
• Basically, find subgoals based on the same
  distance minimization as for the god-object.
• At each subgoal, all the planes that go through
  that point are potential constraints. The minimum
  set of active constraints is selected.
• If the subgoal is in free space, set as new
  subgoal the HIP. The path might intersect the
  C-obstacles. Add the first plane intersected as a
  constraint and the intersection point as the
  current subgoal.
• The process ends when the virtual proxy becomes
  stable.

22
3DOF Haptics Virtual proxy
HIP(t)
Perform collision detection between the path of
the HIP and the C-obstacles
HIP(tDt)
23
3DOF Haptics Virtual proxy
HIP(t)
Set the subgoal and the constraint plane(s)
HIP(tDt)
24
3DOF Haptics Virtual proxy
HIP(t)
Find a new subgoal using the active planes and
the minimization based on Lagrange multipliers
HIP(tDt)
25
3DOF Haptics Virtual proxy
HIP(t)
Since the subgoal is in free space, drop the
constraints, set the HIP as the new subgoal and
perform collision detection between the path and
the C-obstacles
HIP(tDt)
26
3DOF Haptics Virtual proxy
HIP(t)
Recompute subgoal with new constraints
HIP(tDt)
27
3DOF Haptics Virtual proxy
HIP(t)
The path to the new subgoal intersects another
plane, so this is added to the set of constraints
HIP(tDt)
28
3DOF Haptics Virtual proxy
HIP(t)
Compute active constraints (in 2D there are only
2) and find subgoal
For this example, this is the final position of
the virtual proxy
HIP(tDt)
29
3DOF Haptics Virtual proxy

• Quadratic programming approach
• The constraint planes define an open convex
  region (bounded by the plane at infinity).
• The function to minimize is the distance from the
  haptic device (HIP) to the new subgoal (VPi1)
• The translation from the current location to the
  new subgoal cannot intersect the constraint
  planes. Define linear constraints based on the
  normals of the planes.

Quadratic function
Linear constraints
30
3DOF Haptics Virtual proxy

• Special case simplified approach.
• Lets look at the convex region defined by the
  constraint planes

31
3DOF Haptics Virtual proxy

• Normals (n) become points (-n) in a dual space
  (Gauss map).
• The plane at infinity is the origin (o).
• Points (-n) are joint together if the associated
  planes intersect.
• The position of the virtual proxy also has a
  dual
• The vertices of the closest feature to P are the
  duals of the active constraint planes that are
  used for the minimization with Lagrange
  multipliers.

32
3DOF Haptics Virtual proxy
33
3DOF Haptics Virtual proxy

• Force output PD (proportional- derivative)
  control. Produces a force that will try to keep
  the VP and the HIP at the same position.

VP
F
F
Simulation
PD control
Actuators
User
HIP
Its like a springdamper, but the authors look
at it from a control engineering approach
34
3DOF Haptics Additional features

• Force shading by Basdogan et al., 1997.
• Interpolate per-vertex normals using baricentric
  coordinates (same as Gouraud shading)
• Effect edges and vertices seem rounded

35
3DOF Haptics Additional features

• Force shading by Ruspini et al., 1997.
• Modify the position of the VP.
• A subgoal (HIP) is computed changing the normal
  of the plane.
• This subgoal replaces the HIP, and the final VP
  is computed.

36
3DOF Haptics Additional features

• Other effects by Ruspini et al., 1997.
• Friction.
• Textures.
• Deformation.
• All of them are achieved performing operations
  that modify the position of the VP.

37
3DOF Haptics Additional features

• Friction by Hayward and Armstrong, 2000.
• Model friction as an elasticity between the
  actual contact point (x) and a fictitious point,
  called adhesion point (w).
• The adhesion point remains static or follows the
  actual contact depending on the state (sliding,
  static)

x
w
38
3DOF Haptics H-Collide

• What about the collision detection?
• Spatial uniform decomposition of space. Locate
  the path traversed by the probe in that grid,
  using a hash table.
• Test the path against an OBBTree. Optimized test.
• Use frame-to-frame coherence, caching the
  previous contact, and perform incremental
  computation.
• When the leaf intersects, compute the surface
  contact point (SCP). Pass this to GHOST (haptic
  rendering library used by the Phantoms).

39
3DOF Haptics H-Collide
Ray Vs. OBBTree test
Spatial partitioning with hash table
SCP computation
SCP
40
6DOF Haptics Intro

• Output 3D force 3D torque
• For applications related to manipulation.
• Assembly and maintenance oriented design. Removal
  of parts from complex structures.
• Typical problem peg-in-the-hole.

There is a net torque
41
6DOF Haptics Intro

• Formulate the problem as a rigid body simulation
  problem and output to the user the forces exerted
  on the object. Why not?
• Too expensive
• Constantly facing a situation of resting
  contacts.
• Too much time performing collision detection to
  find non-penetrating situations.
• Solve the linear complementarity problem.
• Theoretically, it seems like the natural
  extension of the virtual proxy, though.

42
6DOF Haptics Intro
The position of the center of mass (com) and the
orientation cannot be governed directly by the
haptic device. Interpenetrations would be
unavoidable.
43
6DOF Haptics Contacts

• We cannot afford to compute exact contacts.
• Allow tolerances. Consider as valid contact
  whatever is inside a distance tolerance.
• Outward offset surfaces are better. Distance is
  cheaper to compute than penetration.

44
6DOF Haptics Major approaches

• All the approaches suffer from the fact that its
  an output-dependant problem.
• Point cloud Vs. voxelized model. It can be linear
  on the number of points. The number of points
  grows quadratically with the resolution.
• Polygonal models. At least linear on the number
  of convex pairs inside the tolerance.
• NURBS. Very complex exact distance computation
  between NURBS surfaces.

45
6DOF Haptics Voxel sampling

• McNeely et al., 1999.
• Voxelize all the models in the scene.
• For the haptic object, represent each voxel by a
  point ? Point Shell.
• Test points against voxels of the scene.
• Compute force at each point that penetrates an
  object.
• Add up all the force and torque, and apply them
  to the user through virtual coupling.

46
6DOF Haptics Voxel sampling

• All the objects are voxelized.
• For the haptic object, store points at the center
  of the voxels, with inward normals determined by
  the polygonal model.
• For the objects in the scene, create the voxmap,
  a 512-tree with 3 levels (the 512-tree is an
  extension of the octree, with 512 children per
  node, cube of 888). The 512-tree optimizes
  memory.
• The initial implementation is only for static
  objects.

47
6DOF Haptics Voxel sampling
Original object
Voxels
Pointshell
48
6DOF Haptics Voxel sampling

• All the voxels in the voxmap store info about
  their position in the object interior, surface,
  proximity or free space.

49
6DOF Haptics Voxel sampling

• To start the collision query, all the pointshell
  has to be transformed to the local coordinates of
  the voxelized objects.
• Points that are in the surface or proximity
  voxels are considered as contacts.

50
6DOF Haptics Voxel sampling

• Forces at each contact are computed according to
  the tangent-plane force model.
• Contacts are valid if dlt0 (penetration).
• If penetration to the interior happens, the
  contact is missed. One might think of computing a
  distance field in the interior, but the distance
  depends on the normal N.

51
6DOF Haptics Voxel sampling

• The force is continuous in direction, but
  discontinuous in magnitude.
• The virtual coupling (see later) is supposed to
  filter these discontinuities.

52
6DOF Haptics Voxel sampling

• Force per contact
• Add all the forces together.
• Several contacts act as a single contact with
  higher stiffness.
• Limit the total stiffness
• There can still be problems of irregular
  distribution of stiffness.

If nlt10
If ngt10
53
6DOF Haptics Voxel sampling

• Irregular distribution of stiffness.
• Geometry-driven instabilities

F2F1 ? d25d1 Risk of penetration!
Object bounces from one side to the other. More
common due to torque on long objects.
54
6DOF Haptics Voxel sampling

• Penetration has to be avoided.
• Pre-contact braking forces. Strong viscous force
  applied in the proximity layer.
• The viscosity is adjusted so that all the kinetic
  energy is dissipated in a distance equal to the
  width of the layer.

55
6DOF Haptics Voxel sampling

• Virtual coupling, see Adams and Hannaford, 1998.
• Spring-damper between the position and
  orientation of the haptic device and the position
  and orientation of the object.
• Filters discontinuities, to enhance stability. We
  can run into numerical integration problems,
  though.

56
6DOF Haptics Polygonal models

• Gregory et al., 2000 Kim et al., 2002.
• Convex decomposition of polygonal models.
• Collision detection (modified SWIFT), returning
  all pairs of convex pieces within a tolerance.
  Handle shallow penetrations.
• Cluster contacts.
• Combination of penalty-based restoring forces and
  impulsive response.

57
6DOF Haptics Polygonal models

• SWIFT, by Ehmann and Lin, 2001.
• Hierarchy of convex pieces.
• Voronoi marching between pairs of pieces. Travel
  down if the distance is less than the tolerance.

58
6DOF Haptics Polygonal models

• Modifications to SWIFT
• Return all pairs of pieces inside the tolerance
  (group that contains the set of local minima of
  the distance function).
• Handle shallow penetrations, computing
  penetration depth between convex pieces. Check
  for pairs of features with overlapping normals.

59
6DOF Haptics Polygonal models

• Convex decomposition
• Hard problem. The decomposition might not be
  efficient. Explosion of pairs O(n4).
• Surface decomposition ? Convex patches, completed
  with virtual faces. Linear size.

60
6DOF Haptics Polygonal models

• Surface decomposition good for finding global
  minimum in disjoint situation. Unique pair of
  features, both original.
• Problems for local minima and penetration. The
  features might be virtual.
• If a single face is a convex piece itself,
  penetrating contacts might be missed.

61
6DOF Haptics Polygonal models

• Cluster contacts.
• Find unique contact (distance, normal and point),
  averaging contacts in a neighborhood.
• Weighted average, using distance as the weight.
• This solves the problem of the irregular
  distribution of stiffness.
• Pretty smooth varying forces in practice, without
  need of virtual coupling.
• Additional features (texture, friction) can be
  rendered in a per-contact basis.

62
6DOF Haptics Unsolved problems

• Generalization of Virtual Proxy to 6-DOF haptics
• Haptic Texture Synthesis
• Applications
• Endoscopic sinus surgery
• 3D Painting with deformable brushes
• 3D deformable objects

63
References

• R. J. Adams and B. Hannaford. A Two-port
  Framework for the Design of Unconditionally
  Stable Haptic Interfaces. In Proc. of IEEE/RSJ
  Int. Conference on Intelligent Robots and
  Systems, 1998.
• C. Basdogan, C. H. Ho and M. Srinivasan. A
  Ray-based Haptic Rendering Technique for
  Displaying Shape and Texture of 3D Objects in
  Virtual Environments. In the Proc. of ASME
  Dynamic Systems and Control Division, 1997.
• G. Burdea. Force and Touch Feedback for Virtual
  Reality. John Wiley and Sons, 1996.
• S. A. Ehmann and M. C. Lin. Accurate and Fast
  Proximity Queries between Polyhedra Using Surface
  Decomposition. In Proc. of Eurographics, 2001.
• A. Gregory, M. C. Lin, S. Gottschalk and R. M.
  Taylor II. H-Collide A Framework for Fast and
  Accurate Collision Detection for Haptic
  Interaction. In Proc. of IEEE Virtual Reality
  Conference, 1999.
• A. Gregory, A. Mascarenhas, S. Ehmann, M. C. Lin
  and D. Manocha. Six Degree-of-Freedom Haptic
  Display of Polygonal Models. In Proc. of IEEE
  Visualization, 2000.

64
References

• V. Hayward and B. Armstrong. A New Computational
  Method of Friction Applied to Haptic Rendering.
  In Lecture Notes in Control and Information
  Sciences, Vol. 250, Springer-Verlag, 2000.
• Y. J. Kim, M. A. Otaduy, M. C. Lin and D.
  Manocha. Six-Degree-of-Freedom Haptic Display
  Using Localized Contact Computations. To appear
  in Proc. of Haptic Symposium, 2002.
• W. R. Mark, S. C. Randolph, M. Finch, J. M. Van
  Verth, and R. M. Taylor II. Adding Force Feedback
  to Graphics Systems Issues and Solutions. In
  Proc. of ACM SIGGRAPH, 1996.
• W. A. McNeely, K. D. Puterbaugh and J. J. Troy.
  Six Degree-of-Freedom Haptic Rendering Using
  Voxel Sampling. In Proc. of ACM SIGGRAPH, 1999.
• D. C. Ruspini, K. Kolarov, and O. Khatib. The
  Haptic Display of Complex Graphical Environments.
  In Proc. of ACM SIGGRAPH, 1997.
• C. Zilles and K. Salisbury. A Constraint-based
  God Object Method for Haptic Display. In Proc. of
  IEEE/RSJ Int. Conference on Intelligent Robots
  and Systems, 1995.