Advanced Granular Synthesis for Next-Generation Games

1
(No Transcript)
2
Advanced Granular Synthesis for Next-Generation
Games

• Game Developer's Conference 2007
• San Francisco
• Leonard J. Paul
• Vancouver Film School
• Game Audio Instructor
• info at VideoGameAudio.com
• 604-685-5808 x4035

3
Overview

1. How to utilise granular for speech, sound
   effects, music and engines
2. Prototyping granular synthesis using Pure Data
   and Open Sound Control
3. How to add granular synthesis to your sound driver

4
1)

• Using granular for
• speech
• sound effects
• music and
• engines

5
(No Transcript)
6
Game Parameters

• Granular parameters driven by game state
• (ex. RPM -gt pitch)
• Physics allow for continuous control over
  playback of sample
• (ex. ragdoll -gt foley)
• Input parameters need to be carefully tuned to
  drive synthesis
• (ex. instrument played by novice)

7
(No Transcript)
8
Speech

• Time warping speech for variation
• Change formants of voice
• Realtime granular effects to change character of
  voice

9
Sound Effects

• Crossfade morph via grains
• Spacialization in 5.1/7.1
• Contact physics for sliding/scraping
• Add variation to existing sounds

10
(No Transcript)
11
Music and Ambiences

• Stretch a sound for a granular drone
• Tempo-sync grains for interesting gating effects
• Merge ambiences by selecting grains from multiple
  files
• Making an ambience sound more musical and abstract

12
(No Transcript)
13
Coding PD Objects

• FlexT is a cross-platform method of generating
  custom C PD objects http//grrrr.org/ext/flext/
  
• Easily possible to port part of the Synthesis
  Toolkit (STK) to PD using Code Blocks or any
  other DSP code

14
Flext (gain object code)

• class gain public flext_dsp
• FLEXT_HEADER(gain,flext_dsp)
• float factor
• gain() factor(0)
• AddInSignal("In") // add signal inlet
• AddInFloat("Gain")
• AddOutSignal("Out") // add signal outlet
• FLEXT_ADDMETHOD(1,m_gain)
• virtual void m_signal(int n,t_sample const
  insigs,t_sample const outsigs)
• const t_sample in insigs0
• t_sample out outsigs0
• while(n--) (out) (in)factor
• void m_gain(float db) factor
  pow(10.f,db0.05f)
• FLEXT_CALLBACK_F(m_gain)

15
OSC Layer for Prototyping

• Add an Open Sound Control (OSC) layer to your
  game to communicate with a rapid prototyping
  audio environment such as PD
• Example of using OSC with a Half-life 2 Source
  Mod
• Possible to compile PD to a language such as Java
  (pd2j2me)

16
OSC Code for Sending Msg

• include "osc/OscOutboundPacketStream.h"
• include "ip/UdpSocket.h"
• define ADDRESS "127.0.0.1" // localhost but can
  be any address
• define PORT 7000
• define OUTPUT_BUFFER_SIZE 1024
• int main(int argc, char argv)
• UdpTransmitSocket transmitSocket(
  IpEndpointName( ADDRESS, PORT ) )
• char bufferOUTPUT_BUFFER_SIZE
• oscOutboundPacketStream p( buffer,
  OUTPUT_BUFFER_SIZE ) // create buffer
• p ltlt oscBeginMessage( "/test" ) ltlt "hi
  world" ltlt oscEndMessage // message
• transmitSocket.Send( p.Data(), p.Size() ) //
  send message

17
Adding OSC to Half-life 2

• Download the OSCpack library from
• http//www.audiomulch.com/rossb/code/oscpack/
• Download Visual C 2005 Express, Half-Life 2 and
  create a new mod
• Add the OSC files to the mod source (some
  tweaking needed) and modify SoundEmitter.cpp etc.
• This is a similar process to adding OSC to your
  own sound driver

18
(No Transcript)
19
Granular Sound Driver

• PD C code can be ported to the next-gen
  consoles as the consoles are primarily software
  based
• Synthesis should be creatively used in
  conjunction with samples for sound design
• Advantage is that real-time control over
  synthesis parameters is from game parameters

20
Granular Code Examples

• Ross Bencina Granular Delay Line
• Uses Borland C Builder
• http//www.audiomulch.com/rossb/
• Synthesis Toolkit (STK) Granulate
• Can use flext to port to PD
• http//ccrma.stanford.edu/software/stk/
• SyncGrain for PD (from Csound)
• Ported using flext
• http//footils.org/cms/pms/

21
Related Research

• Concatenative Sound Synthesis
• Sound "Mosaic" Analysis Resynthesis
• Uses grains from one sample to approximate grains
  of a second sample
• www.mat.ucsb.edu/b.sturm/sand/VLDCMCaR/VLDCMCaR.h
  tml

22
Related Research

• Natural Grain Resynthesis
• Make grains that preserve transients
• http//www.cs.ubc.ca/reynald/naturalgrains.html

23
(No Transcript)
24
Example

• Example of gameplay utilizing granular synthesis

25
Questions?

• Email info-at-VideoGameAudio.com
• Web www.VideoGameAudio.com