Advanced Granular Synthesis for NextGeneration Games

1
(No Transcript)
2
Advanced Granular Synthesis for Next-Generation
Games

• Game Developer's Conference 2007
• San Francisco
• Leonard J. Paul
• Vancouver Film School
• Game Audio Instructor
• info at VideoGameAudio.com
• 604-685-5808 x4035

3
Overview

• How to utilise granular for speech, sound
  effects, music and engines
• Prototyping granular synthesis using Pure Data
  and Open Sound Control
• How to add granular synthesis to your sound driver

4
1)

• Using granular for
• speech
• sound effects
• music and
• engines

5
Granular Parameters

• Selection order (fwd/rev or freeze)
• Pitch shift
• Amplitude range
• Spatialization / panning
• Grain duration
• Grain density (grains/sec, of grain voices)
• Envelope (ASR shape or windowing function)
• DSP effect (reverb, filtering...)
• Feedback amount (for granular delay lines)

6
Game Parameters

• Granular parameters driven by game state
• (ex. RPM -gt pitch)
• Physics allow for continuous control over
  playback of sample
• (ex. ragdoll -gt foley)
• Input parameters need to be carefully tuned to
  drive synthesis
• (ex. instrument played by novice)

7
Game Audio Data Diagram
8
Speech

• Time warping speech for variation
• Change formants of voice
• Realtime granular effects to change character of
  voice

9
Sound Effects

• Crossfade morph via grains
• Spacialization in 5.1/7.1
• Contact physics for sliding/scraping
• Add variation to existing sounds

10
Engines

• Add as a layer for variation
• Pre-segmentation and playback via RPM/load band

11
Music and Ambiences

• Stretch a sound for a granular drone
• Tempo-sync grains for interesting gating effects
• Merge ambiences by selecting grains from multiple
  files
• Making an ambience sound more musical and abstract

12
2)
13
Coding PD Objects

• FlexT is a cross-platform method of generating
  custom C PD objects http//grrrr.org/ext/flext/
  
• Easily possible to port part of the Synthesis
  Toolkit (STK) to PD using Code Blocks or any
  other DSP code

14
Flext (gain object code)

• class gain public flext_dsp
• FLEXT_HEADER(gain,flext_dsp)
• float factor
• gain() factor(0)
• AddInSignal("In") // add signal inlet
• AddInFloat("Gain")
• AddOutSignal("Out") // add signal outlet
• FLEXT_ADDMETHOD(1,m_gain)
• virtual void m_signal(int n,t_sample const
  insigs,t_sample const outsigs)
• const t_sample in insigs0
• t_sample out outsigs0
• while(n--) (out) (in)factor
• void m_gain(float db) factor
  pow(10.f,db0.05f)
• FLEXT_CALLBACK_F(m_gain)

15
OSC Layer for Prototyping

• Add an Open Sound Control (OSC) layer to your
  game to communicate with a rapid prototyping
  audio environment such as PD
• Example of using OSC with a Half-life 2 Source
  Mod
• Possible to compile PD to a language such as Java
  (pd2j2me)

16
OSC Code for Sending Msg

• include "osc/OscOutboundPacketStream.h"
• include "ip/UdpSocket.h"
• define ADDRESS "127.0.0.1" // localhost but can
  be any address
• define PORT 7000
• define OUTPUT_BUFFER_SIZE 1024
• int main(int argc, char argv)
• UdpTransmitSocket transmitSocket(
  IpEndpointName( ADDRESS, PORT ) )
• char bufferOUTPUT_BUFFER_SIZE
• oscOutboundPacketStream p( buffer,
  OUTPUT_BUFFER_SIZE ) // create buffer
• p ltlt oscBeginMessage( "/test" ) ltlt "hi
  world" ltlt oscEndMessage // message
• transmitSocket.Send( p.Data(), p.Size() ) //
  send message

17
Adding OSC to Half-life 2

• Download the OSCpack library from
• http//www.audiomulch.com/rossb/code/oscpack/
• Download Visual C 2005 Express, Half-Life 2 and
  create a new mod
• Add the OSC files to the mod source (some
  tweaking needed) and modify SoundEmitter.cpp etc.
• This is a similar process to adding OSC to your
  own sound driver

18
3)
19
Granular Sound Driver

• PD C code can be ported to the next-gen
  consoles as the consoles are primarily software
  based
• Synthesis should be creatively used in
  conjunction with samples for sound design
• Advantage is that real-time control over
  synthesis parameters is from game parameters

20
Granular Code Examples

• Ross Bencina Granular Delay Line
• Uses Borland C Builder
• http//www.audiomulch.com/rossb/
• Synthesis Toolkit (STK) Granulate
• Can use flext to port to PD
• http//ccrma.stanford.edu/software/stk/
• SyncGrain for PD (from Csound)
• Ported using flext
• http//footils.org/cms/pms/

21
Related Research

• Concatenative Sound Synthesis
• Sound "Mosaic" Analysis Resynthesis
• Uses grains from one sample to approximate grains
  of a second sample
• www.mat.ucsb.edu/b.sturm/sand/VLDCMCaR/VLDCMCaR.h
  tml

22
Related Research

• Natural Grain Resynthesis
• Make grains that preserve transients
• http//www.cs.ubc.ca/reynald/naturalgrains.html

23
Related Research

• Pure Data Compiler (pd2j2me)
• Compile PD code to Java
• http//www.uow.edu.au/mh675/publications/nime_200
  5.pdf

24
Example

• Example of gameplay utilizing granular synthesis

25
Questions?

• Email info-at-VideoGameAudio.com
• Web www.VideoGameAudio.com