Java Audio Synthesis System http:www.cs.ubc.cakvdoeljass - PowerPoint PPT Presentation

Title: Java Audio Synthesis System http:www.cs.ubc.cakvdoeljass


1
Java Audio Synthesis System http//www.cs.ubc.ca/
kvdoel/jass
Kees van den Doel and Dinesh K. Pai Department
of Computer Science, University of British
Columbia
SUMMARY JASS is a unit generator based audio
synthesis programming environment written in
pure Java. JASS is freely available with full
source code. The environment is based on a
foundation structure consisting of a small number
of Java interfaces and abstract classes, and a
potentially unlimited number of unit generators,
which are created by extending the abstract
classes and implementing a single method.
Filter-graphs, sometimes called patches, are
created by linking together unit generators
in arbitrary complex graph structures.
Patches can be rendered in real-time with
special unit generators that communicate with
the audio hardware, which we have implemented
using the JavaSound API. We used JASS to develop
algorithms for real-time synthesis of realistic
sound effects for interactive simulations (e.g.,
games) and animation. These sound effects are
produced automatically, from 3D models using
dynamic simulation and user interaction. The
algorithms are efficient, physically-based, and
can be controlled by users in natural ways.
Audio is simulated using modal vibration models
(determined by material and shape) driven by
contact forces (determined by surface
roughness). The contact forces can be custom
designed or can be computed from simulations.
InOut UG Template
MODAL MODEL Vibrations of solids are modeled as
modal resonator banks. Parameters are computed,
measured, or set manually. Mode shapes determine
timbral variations.
Source
The InOut Unit Generator produces audio buffers
and receives audio buffers from external Sources.
Getbuffer() retrieves an audio-buffer and updates
attached Sources.
Out
Sink
source1
source2
time
1750
buf

Source1 buf1 Source2 buf2 Source3 buf3
source3
computeBuffer()
getBuffer(t)
Simulation of Models
DYNAMICS SIMULATOR The simulator computes the
parameters needed to drive the audio and the
graphics. They include the contact forces for
sustained contact, impulsive forces for
collisions, the relative surface velocities, and
the velocities of the contact points on the
surfaces. The contact parameters are provided to
JASS at the rate of simulation for
micro-simulation of contact interactions, while
the system state is interpolated at frame rate
for rendering graphics.
SOUND SYNTHESIS Audio is rendered with JASS
synthesis toolkit in real-time from modal
vibrations and sent to audio hardware. The user
interacts with the simulation by looking at the
display and listening to the sounds and also
provides input through mouse, joystick, or other
more immersive means.
CONTACT FORCES From the local contact physics we
determine contact speed and velocity. With a
surface roughness model this gives us the
audio-force, which drives surface vibrations.
Contact Force Model UG
Resonance Model UG
Source Player UG
JASS
Unit Generator Templates
start()
v, F, roughness
p, damping, loadModel(file)
Source
Out
UG1
Unit Generators are subclasses of templates Out,
In, and InOut, which implement inter-faces Source
or Sink.
Simulator
InOut
UG2
user input
positions
Sink
In
UG3
v
F
User
Graphic Display
Legend
p
display
Class
Interface
extends