Medical Simulation

1
Medical Simulation

• Talk by Lisa Lyons

2
Surgery Simulation Requirements

• Realistic visualization of internal organs
• Organs react realistically in real time to
• User interactions
• Environmental restrictions
• Organs react to typical surgeons gestures
  through geometric and topological modifications

3
Surgery Simulation Grouping

• First generation
• Only deal with geometric nature of human anatomy
• Second generation
• permit physical interactions with anatomy
• Include needle-type, exploration-type, catheter
  installation-type simulators as well as
  simulators that permit training in only one task
  and full simulators
• Third generation
• consider functional nature of organs

4
Outline

1. Physical Modeling
2. Reduction of Computing Time
3. Collision Detection
4. Example Systems
5. Results and Conclusion

5
Anatomical Model of the Liver

• Data set consists of about 180 slices of frozen
  human tissue that has been put through CT scan
• Enhance contrast
• Apply edge detection
• Semi-automatic deformable models ? binary images
• Stack images to form 3D binary image Montagnat,
  1997

6
Simplex Meshes

• Better than marching cubes avoids staircase
  effects
• Developed by Delingette to represent 3D objects
  Delingette, 1994
• Adaptable (figure to right)
• Working on a method to extract liver models from
  CT images

7
Force Feedback

• How physically realistic the model is correlated
  with how realistic force feedback is
• Model deforms with surgeons motion
• Contact force may be computed from deformation
• Force generated back to surgeon through
  mechanical actuators

8

• Method uses linear elasticity as an approximation
  for tissue deformation
• Let the configuration of an elastic body be
  defined as O
• A field of volumetric and surface forces f acts
  on the body so it has a new configuration O
• We want the displacement field u which associates
  the initial configuration of any particle with
  its final configuration
• Use FEM Lagrange elements of type P1 Bathe,
  1996
• Formulate the problem as a linear system
• Where K is the 3n by 3n stiffness matrix and n
  is the number of mesh vertices (more on this in a
  minute)

9

• Only thing we know is endoscope position
• must use displacement not force constraints
• Given some displacements between the surgical
  tool and the body, we can find
• Force on end effecter
• Global deformation
• Now we use variational formulation and Lagrange
  multipliers to minimize
• Include constraints u u
• Solving for ?i gives the opposite of the
  necessary forces to impose the displacement u
• See Appendix A Cotin, 1999 for full derivation

10
Stiffness matrix containing 3X3 mini-matrix of
stiffness information for each node
Matrix composed of a 3X3 identity matrix for each
constrained segment (k)
Forces required to obtain desired state
Desired displacements of k nodes
11
Linear Representation

• In theory, this behavior is only physically
  correct for small displacements
• Force feedback limits the range of deformations
• Feedback force on surgeons hand will increase as
  deformation increases

12
Quasi Non-Linear Representation

• Mix of linear representation and empirical
  results using a cylindrical piece of brain tissue
• Chinsei, 1997 found that deformation depends on
  loading speed and is nonlinear

13
Outline

1. Physical Modeling
2. Reduction of Computing Time
3. Collision Detection
4. Example Systems
5. Results and Conclusion

14
Computation Time

• Number of mesh vertices has high impact
• Makes matrices larger
• Must use speedups
• Cannot make necessary calculations in real-time

15
Pre-Computation Algorithm

• Specify a set of nodes to remain fixed
• Dont have to set all three dof
• For every free node k and degree of freedom on
  the surface, emplace an elementary displacement
  constraint (d)
• Denote this as
• Compute the displacement of every free node n in
  the mesh with respect to every node k
• Store as set of 3X3 tensors
• Compute elementary force at each constrained node
  k
• Store as 3X3 tensors

16
Solving The Linear System

• Must be solved 3m times where m is the total
  number of free nodes inside the tetrahedral mesh
• Can take anywhere from a few minutes to several
  hours

17
Linear Elasticity

• For any n where k ? n, the relation between n and
  k is
• Superposition may be used to find the total
  displacement of a node but some modifications
  must be made

18

• Use tensors of deformation found in preprocessing
  to generate a vector of modified constraints
• where
• and

19

• From this, we can find the displacement of any
  node
• The force that must be applied to each node k to
  produce these displacements is

20
Quasi Non-Linear Elastic Deformations

• Computing times for a realistic looking liver
  model

21
Outline

1. Physical Modeling
2. Reduction of Computing Time
3. Collision Detection
4. Example Systems
5. Results and Conclusion

22
Collision Detection

• Work discussed so far uses bounding boxes with a
  hash table
• We know about these so lets move on to a new
  problem simulating the folds of the intestines

23
Simulating Intestines

• Goal is simulator to allow doctors to practice a
  surgery that involves pulling and folding the
  intestines Raghupathi L. et. al., 2003
• Real problem here is self-collsions
• Complicated by tissue called mesentery
• Connects small intestine and blood vessels

24
Model

• Resting position
• Intestines look like folded curves lying in a
  cylinder
• Mesentery is defined as line segments connecting
  folded intestine to the axis of the cylinder
• Mechanical model uses masses and springs

25
Collisions Between Intestines

• Model intestines like cylinders
• Find distance between their principle axes
• Active pairs
• Local minima satisfying certain distance
  threshold
• Updated every time step
• N additional random pairs of segments also
  generated every time step
• These are tested and thrown out if they are over
  the threshold or already represent a minimal pair

26
Mesentery Collisions

• Complexity would be too high for real-time
  without approximation
• Dont consider mesentery-mesentery interactions
• Adaptive convergence
• Replace segment S1 by closest neighbor S to S2
  and then replace S2 with neighbor closest to S
• When collision occurs, recursive search begins
  across neighbors

27
Results and Demo Video
28
Outline

1. Physical Modeling
2. Reduction of Computing Time
3. Collision Detection
4. Example Systems
5. Results and Conclusion

29
GeRTiSS

• The Generic Real Time Surgery Simulator
  Monserrat et al., 2003

30
Scene Generator

• Allows user to select tools and organs needed
• Systems contains modeling parameters for a
  variety of organs
• Mass-spring model
• Boundary element based model (BEM)

31
Scene Generator

• Tools
• Loading organs
• Establishing input points for instruments
• Associating different physical properties with
  organs
• Establishing boundary conditions
• Linking tissues
• Adding special tissues
• Associating textures to organs

32
Surgery Simulator

• Takes a scene and allows user to train
• User can have interaction with organs
• Cut
• Cauterize
• Drag
• Clip
• User can exchange instruments
• User is assessed at the end based on how many
  incorrect actions were taken

33
Results

• Use 450 MHz Pentium III with 256 MB memory
• Computational Costs

34
User Interface for Laproscopy
35
Haptics

• For good visual image 15Hz refresh rate
• For good haptic stimulus 500 Hz refresh rate
• Use a PC cluster to solve this
• Cost of force feedback devices makes simulator 4X
  more expensive than without

36
Cataract Surgery Simulation

• Surgery aims to extract cataract and replace it
  with intraocular lens Agus et al., 2006
• Training is important
• Simulation allows
• Flexibility
• Gradual increase in difficulty
• Exposure to rare events
• Quantification of performance

37
The Procedure

• Phacoemulsification breaking hardened lens into
  fragments and removing them with a small sucker
  using the phacoemulsificator
• Create z-shaped corneal tunnel
• Capsulorhexis removing the anterior capsule to
  uncover the upper surface of the crystalline

38
Methods

• Decoupled simulation
• Fast subsystem for surgical instrument tracking
  and slower one for visual feedback
• Slow subsystem does global simulation and
  interaction of devices and eye
• Slow subsystem can be further broken into
  individual visual effects
• Force feedback is useless in this surgery
• Must use eye globe visualization
• Conjugate gradient to minimize energy constraints
  gives equilibrium position
• Rotate to reduce deformation

39
Capsulorhexis Simulation

• Use triangular mesh with a mass-spring network
  mapped over it
• Mass particles may be anchored, scripted or free
• Gravity, viscosity and springs contribute to
  acceleration
• Weak springs simulate sticking effects
• Solve ODE using semi FSAL (First Same as Last)
• Velocity found using implicit method and feedback
  on position is computed explicitly
• Correction routine applied after each step to
  correct position and velocity as required by
  constraints
• Tearing breaking overextended springs

40
Phacoemulsification Simulation

• Lens as collection of simplices
• Tetrahedron mesh with particles placed at
  barycenters
• Links connecting particles maintained for
  rendering and determining independent particles
• Photoemulsificator modeled by eroding particles
  in a zone of influence
• Employ Russian roulette scheme to decide which
  particles to erode
• When particles are removed, simplicial mesh is
  updated
• Idea is to replace energies by geometric
  constraints and forces by distance from current
  position to goal
• Each connected subset of points is associated
  with a point cloud
• Shape matching with undeformed rest state to
  determine goal positions

41
Results
42
Outline

1. Physical Modeling
2. Reduction of Computing Time
3. Collision Detection
4. Example Systems
5. Results and Conclusion

43
Surgical device with force feedback simulation
Visual feedback
44
Where is the future?
45
Appendix A Collision Response

• Tried penalty and constraint methods but
  stability of the system was reduced
• Instead alter displacement velocities to avoid
  penetration

46
Appendix A (cont.)

• Interpolating
• Need force f f so we have
• New velocities are
• Substituting we get

47
Appendix A (cont)

• Solving for f gives
• Condition for avoiding penetration takes radii
  into account
• The force required to change the positions of the
  endpoints to satisfy these conditions is

48
References

1. Marco Agus, Enrico Gobbetti, Giovanni Pintore,
   Gianluigi Zanetti, and Antonio Zorcolo. Real-time
   Cataract Surgery Simulation for Training. In
   Eurographics Italian Chapter Conference.
   Eurographics Association, 2006.
2. K.-J. Bathe, Finite Element Procedures. Prentice
   Hall, 1996.
3. K. Chinsei and K. Miller, Compression of Swine
   Brain Tissue Experiment In Vitro, J. Mechanical
   Eng. Laboratory, pp. 106-115, 1997.
4. S. Cotin, H. Delingette, and N. Ayache. A Hybrid
   Elastic Model allowing Real-Time Cutting,
   Deformations and Force-Feedback for Surgery
   Training and Simulation. The Visual Computer,
   16(8)437-452, 2000.
5. Cotin, S. Delingette, H. Ayache, N., "Real-time
   elastic deformations of soft tissues for surgery
   simulation," Visualization and Computer Graphics,
   IEEE Transactions on , vol.5, no.1, pp.62-73,
   Jan-Mar 1999
6. H. Delingette, Simplex Meshes A General
   Representation for 3D Shape Reconstruction,
   Technical Report 2214, INRIA, Mar. 1994.
7. Y.C. Fung, Biomechanics-Mechanical Properties of
   Living Tissues, second ed. Springer-Verlag, 1993.
8. Carlos Monserrat, Oscar López, Ullrich Meier,
   Mariano Alcañiz Raya, M. Carmen Juan Lizandra,
   Vicente Grau GeRTiSS A Generic Multi-model
   Surgery Simulator. IS4TH 2003 59-66
9. J. Montagnat and H. Delingette, Volumetric
   Medical Images Segmentation Using Shape
   Constrained Deformable Models, Proc. First Joint
   Con5 CVRMed-MRCAS 97, J. Troccaz, E. Grimson,
   and R. Mosges, eds. Mar. 1997.
10. M. Moore and J. Wilhelms, Collision Detection
    and Response for Computer Animation, Computer
    Graphics (SIGGRAPH 88), vol. 22, pp. 289-298,
    Aug. 1988.
11.   Laks Raghupathi, Laurent Grisoni, Fran?ois
    Faure, Damien Marchal, Marie-Paule Cani,
    Christophe Chaillou, "An Intestinal Surgery
    Simulator Real-Time Collision Processing and
    Visualization," IEEE Transactions on
    Visualization and Computer Graphics, vol. 10, no.
    6, pp. 708-718, November/December, 2004.