GDC 2005

1
(No Transcript)
2
Better ToolsforSound Designers on
PlayStation3through Open Architecture Designs
Michael Kelly, Senior Audio Engineer, Sony
Computer Entertainment Europe
3
What does that mean?

• On PlayStation 3
• Very powerful audio engine
• No tools to fully exploit it
• Open architecture
• Documented
• Pluggable
• Open Source
• Published file format

4
Overview

• Introduce PS3 Audio Engine
• Reasons for rethinking audio tools
• Discuss our approach
• Show
• How this model benefits game-audio

5
MultiStreamTM Introduction

• The de-facto PS3 Audio Engine
• Entirely software based
• Therefore its optional
• Probably the most powerful and flexible audio
  engine in the world!

6
MultiStream RoutingExample Voices to master

• Up to 512 voices

MASTER
Voice0
Voicen
Voice511
7
MultiStream RoutingExample All busses to Master

• Multiple busses to master

Voice
MASTER
Bus 0
Voice
Bus n
Voice
Bus 30
Voice
Voice
8
MultiStream RoutingExample Busses route to
each other

• Asymmetrical configuration

Bus 1
MASTER
Bus 0
Bus 2
Bus 3
Bus 4
9
MultiStream Effects

• Compressor, Multiband Compressor, Softclip
  Distortion, Polynomial Distortion
• Eq on bus, Parametric eq, Filters (15 types)
• FDN reverb, I3DL2, Convolution Reverb
• Pitch Shift/Time Stretch, Vocoder
• Delay, Ring Modulator

10
MultiStream Effects

• Each bus can have 8 effects

Bus
Slot 0Filter
Slot1 Reverb
Slot2 Comp
Slot3 EQ
Slot4 Empty
Slot5 Empty
Slot6 Empty
Slot7 Empty
11
MultiStream Effects

• So can each voice

Voice
Slot0 Stretch
Slot1 Conv.
Slot2Empty
Slot3 Empty
Slot4 Empty
Slot5 Empty
Slot6 Empty
Slot7 Empty
12
MultiStream Effects

• Send and return Voices and busses

Bus 0
Slot 0Filter
Slot1 Reverb
Slot3 EQ
Slot4 Empty
Slot5 Empty
Slot6 Empty
Slot7 Empty
Slot2 Send
Bus 1
Slot 1Filter
Slot2Reverb
Slot3 EQ
Slot4 Empty
Slot5 Empty
Slot6 Empty
Slot7 Empty
Slot 0Return
13
MultiStream RoutingSend and return

• Complicates routing further

Bus 1
MASTER
Bus 0
Bus 2
Bus 3
Bus 4
14
MultiStreamMulti-Channel

• Each voice and bus
• 8 Channels (eg. 7.1 Surround)
• Matrix Routing
• Voice to Bus
• Bus to Bus
• Send to Return
• Sends and returns
• From/To any channel

15
MutliStream Power

• We can run 50 X 2 second impulses
• on a single SPU
• Decode 400 MP3 Streams
• 24kHz / 128kb/s

16
MultiStream Summary

• Endlessly configurable
• Many effect combinations
• 15 effects out of the box
• ATRAC, VAG, MP3 support
• Roll your own Effects
• Roll your own formats
• How do you make a tool to do that?

17
New Tool Design
18
No MultiStream ToolsWhat about SCREAM?
19
SCREAM / MultiStream

• Uses fixed config

MultiStream Bus
Stream0
SRC/Pitch
PEQ
Dist.
Send
Filt
Streamn
SRC/Pitch
PEQ
Dist.
Send
Filt
Stream127
SRC/Pitch
PEQ
Dist.
Send
Filt
20
SCREAM Summary

• Only uses a fraction of MultiStream
• Specific to PlayStation platforms
• Robust but dated

21
New Tool Spec

• Easy to use
• Provide complex routing and effects configuration
• Improve workflow
• Cross-Platform Engine
• PS3, PlayStation 2, PSP
• Xbox 360, PC, MAC

22
Easy to use

• Basic assumption
• -A whole other talk

23
Routing configuration

• Sound Punk Tool (working title)
• Desribe bus routing
• Desribe effects config
• Simplifies MultiStream setup

24
Cross Platform

• Many approaches
• Eg. Use Sound Punk to describe/emulate another
  platform
• Configure MultiStream to be another engine
• Lowest common denominator problem
• Cross platform engines
• Cross platform tools
• Whats the difference

25
Cross PlatformThe traditional approach

• Middleware engine
• Wraps console hardware
• Fixed tools

Programmer
Tool/Audio API
Hardware Engine
26
Cross PlatformLowest common denominator

• In the hardware days

Tool
API
Platform
27
Cross PlatformThe software engine

• XBox360 and PS3
• Very little audio hardware
• Software engine
• Same code
• Except for optimizations

28
Cross PlatformThe Tool Engine

• With each tool you need the runtime engine
• Bound to audio engine
• Solves the lowest common denominator problem
• This gives us a problem
• MultiStream is optional!

29
Cross PlatformSeparate tool Engine

• This is the way it used to be (kind of)
• Good for us
• Our tool works with other peoples engines
• Good for other people because their tools work on
  our engine
• Sound designer and programmer separate choice
• Its not that simple
• How to avoid the LCD problem

30
Separating the tool from the engine
31
Engine Overview

• Abstract the engine
• Configuration
• Triggers
• Param updates

32
Abstraction

• Abstraction model SP-EAL

Tool API
EngineAbstraction
33
Sound PunkDescribe engine config

• Describes the audio engine
• Max voices
• Max busses
• Effects
• Routing Options
• Describes specific configuration
• Bus routing
• Active effects

34
Abstraction Sound Punk

• Platform file describes engine limits
• Voices, busses, effects, etc
• Also describes config
• bus names, source names, formats
• Testing with different well-known APIs

35
Sound Punk Supplied libraries on PS3

• PS3 runtime library layer
• Including source code
• Developer can re-implement for other platforms
  (Wii, Xbox 360)
• Example PC implementation
• Script Engine also as source
• Developer can rebuild for other platforms

36
Lowest common denominator solved

• Everything is abstracted
• Mapping
• Scalability

37
Engine abstraction Bonuses

• Once you abstract the engine, new things for free
• Remote auditioning
• In game preview
• Cross platform

38
Problems

• Voice allocation
• Tool or engine
• Missing effects
• Different effects
• File Formats
• All implementation dependent
• -Accommodate in tool

39
Another approachThe file format
40
Common File format

• Engine abstraction and File Format
• Not mutually exclusive
• Agreed tool-level interchange format
• Standard format for asset data
• Standard format for scripts
• Standard format for parameters

41
Tool choice
42
Common file format

• Advantages
• Sound designer chooses right tool for right job
• Common terminology
• Common expectations
• Should still allow USPs

43
Common File FormatWhere to start?

• Dont reinvent the wheel
• IASIG iXMF
• Interactive Extensible Media Format

44
Common File FormatXMF Folder layout
File 1
Data
Folder
Node
Data
Folder
Folder
Node
Node
Data
Folder
Node
Node
Node
Node
File 2
Data
Data
45
Common File FormatAsset Types

• Media files
• Chunks references to media files
• Cues and Scripts

46
Common File FormatScripting

• Script language defined
• Roll your own
• Everything is extensible

47
Common File FormatExtensibilty

• Metadata
• New script languages
• New anything

48
Problems

• GUI Metadata
• Common file format, common tool?
• Unsupported extensions?
• Corruption
• Inefficient at runtime?

49
Summary

• Rationalising the sound process
• SCEE has two approaches
• Engine Abstraction Config Tool
• Open File Format
• Perfect approach for us
• What about you?
• Improves workflow

50
Industry benefits

• Mature idea of audio engine
• Common terminology
• True choice for developers
• Improved Workflow
• Better sounding games

51
This isnt the end

• (well it is, but..)
• More discussion to follow
• Sony Computer Entertainment Developers
• Events like GDC
• Bodies like IASIG, AES, BBQ
• And the QA