Echo and Bounce

1
Echo and Bounce

• Marc OMorain

2
Presentation

• About 15 Minutes of PowerPoint
• 5 Minutes of video and demonstration
• Please ask questions at any time

3
A Rabbit
4
A Visual Proxy
5
A Physical Proxy
6
An Audio Proxy

• This project will explore the idea of using an
  audio proxy
• A proxy to generate sound

Visual Proxy
Physical Proxy
Audio Proxy
Realistic Movement
Realistic Sound
Realistic Graphics
7
Past Research

• Synthesizing Sounds from Rigid-Body Simulations.
• O'Brien, J. F.Shen, C., Gatchalian,
  (SIGGRAPH 2002)
• Synthesizing Sounds from Physically Based Motion.
• O'Brien, J. F., Cook, P. R., Essl G.,
  (SIGGRAPH 2001)
• FoleyAutomatic Physically-based Sound Effects
  for Interactive Simulation and Animation
• K. van den Doel, P. G. Kry and D. K. Pai,
  (SIGGRAPH 2001)

8
Why?

• Why is this useful?
• No need to record sounds
• Information needed is in physics engine already
• What is new?
• Level of detail
• Using mass springs no pre-computation

9
What is an Audio Proxy

• What is an audio proxy made of?
• Masses and Springs
• Tetrahedra
• How do we build one?

10
Mass Spring System
11
Mass Spring System

• When a spring is at rest it has no resultant
  effect on masses
• When compressed, pushes two masses away from each
  other
• When extended, pulls two masses toward each other

12
3D Meshes

• A triangular mesh can approximate any 3D surface
• Commonly used in graphics
• Just a skin
• No Volume
• No Density

13
Tetrahedra

• A Tetrahedron is a 4 sided shape
• (Triangular based pyramid)
• Made from 4 triangles
• Collection of tetrahedra can approximate any 3D
  volume

14
Mass Spring Tetrahedra

• Create tetrahedra from mass-spring system
• Soft body of any shape can be created
• This is how all objects in the project are
  represented

15
Digital Audio
Time
-
Amplitude
16
Sound Generation

• Sound Generation

17
Sound Generation
Apply A Force
Resultant Vibration
18
Sound Generation

• For each face
• Find displacement at each vertex
• Find average displacement
• Multiply by area
• Wave ampltiude
• (Average displacement area)

19
Vibration to Sound
Time
20
Up Sampling

• Take samples at quite a low granularity
• Fit a hermite curve to the sample data
• Re-sample from curve at higher resolution
• Sample at CD-quality

21
1st Major Problem

• The Sound was crackling
• Reason

• Hermite curve blends between 4 values
• Dont know what is coming in the future
• Assume Zero
• Curve goes below zero

(unsigned short) -0.00007f 65536
22
A Click _at_ 33Hz
65536
16-bit
0
23
Simulation

• Simulation

24
Mass Stability

• Each Mass in the system has a short term memory
  (Last 5 timesteps)
• At each timestep current movement is added to
  memory
• If the mass does not remember moving
• It is stable
• Masses will stay stable until an external force
  acts on them (Newtons 1st Law)

25
Audio Proxy Stability

• Simulating a stable mass is free
• If all the masses are stable, the entire proxy is
  stable
• Simulating a stable proxy is free

26
A Physics Engine

• This is not a physics engine
• All external forces will come from a physics
  engine
• When a collision happens, the physics engine will
  provide
• The Point of collision
• A Force

27
Objects in Model Space

• All objects are kept in model space
• (All at the origin)
• All models are independent from each other
• Simulate all models seperately

28
Levels of Detail

• Levels of Detail

29
Levels of Detail

• Because this project uses
• Up sampling
• Separate Models
• Different timesteps for each model
• Small timestep for close objects
• Larger timestep when further away

30
Different Sampling Rates
Low Sampling Rate
Higher Sampling Rate
31
Levels of Detail

• Simulation time is proportional to
• Number of masses in the system
• Number of springs in the system
• System with s springs and m masses
• Time t per calculation is

32
Levels of Detail
6 Tetrahedra
166 Tetrahedra
24 Tetrahedra
33
Variable Level Of Detail
Listener is far away Low level of detail
34
Variable Level Of Detail
Listener is medium distance - Higher level of
detail
35
Variable Level Of Detail
Listener is very close - Highest level of detail
36
Resonance
37
A Screenshot
38
A Screenshot
Sound Wave (Left and right channels)
39
A Screenshot
Field Of Vision (Blue Cone)
40
A Screenshot
Level of Detail Selection
41
A Screenshot
Click to apply a force
42
Echo and Bounce

• Now for some videos
• and a demonstration