Audio

1
Audio
2
The Basics of Sound

• Sound is mechanical wave that travels in the air.
  It has two primary characteristics
• Amplitude Loudness
• Frequency Pitch. Audible frequency range 20 Hz
  to 20 KHz.
• Two kind of sounds that a computer can make
• Digital sound samples electrical signals created
  by a sound recorder and encodes the sampling data
  in digital format.
• Sampling rate According to sampling theorem, The
  sampling frequency must be at least twice the
  maximum frequency of the function in order to
  accurately reconstruct the function
• Amplitude resolution Number of bits used for
  digitization. 16-bit recording for
  professional-quality sound.
• Synthesized sound is produced through special
  algorithms based on some sort of description.
• MIDI (Musical Instrument Digital Interface)
  contains info about musical tons and timing and
  leaves synthesis to hardware.

3
SDL Audio

• Open the audio device
• Load the audio file
• Play the audio
• Clean up

4
Understanding the runtime behaviour of the SDL
sound subsystem

• A typical SDL audio driver requests two buffers
  at whatever sample size is specified by the
  application, and then pre-fills them with
  silence. Then it double-buffers sound, filling
  one buffer (thanks to the user-specified
  callback) while the audio hardware is reading the
  other.
• For example, let A and B be buffers, both 4kb
  large (1024 samples of 16-bit stereo data)
• A and B are allocated and filled with silence
• Audio hardware is triggered to playback A
• Application gets a callback to fill B
• Audio driver waits for A to finish playing and
  triggers playing B
• Application gets a callback to fill A
• ...and so on...
• This scheme assumes that the application is able
  to fill the buffers in time, which is usually
  true with current CPU and OS combinations.

5
Callbacks
6
DirectX Audio

• DirectX Audio allows you to play both digital and
  synthesized music with all kinds of options,
  including 3D sound.
• DirectX is based on Microsoft COM technology. We
  need to create several COM objects to play music
• performance is the master interface of DirectX
  Audio. It handles the flow of data from the
  source to the synthesizer. You need only one
  performance object for your program.
• A segment is a piece of music. You create one
  segment for a piece of music.
• An audiopath controls the flow of sound. It can
  be viewed as a chain of objects through which
  sound data is streamed. The performance has a
  default audiopath, but you can create additional
  audiopaths, e.g. using 3D audiopath for playing
  3D sound.

7
Steps to Play an Audio File with DirectMusic

• Initialize COM
• Create the performance
• Initialize the Audiopath
• Create the loader
• Load a segment
• Download the band
• Play the segment

8
Initialization

• Initialize COM
• HRESULT CoInitialize(NULL)
• Create the Performance using COM interface
• IDirectMusicPerformance8 g_performance NULL
• CoCreateInstance(
• CLSID_IDirectMusicPerformacnce,
• NULL,
• CLSCTX_INPROC, IID_IDirectMusicPerform
  ance8, (void) g_performance
• )

9
Initialize the Performance

• Initialize the performance and set up a default
  audiopath of a standard type
• g_performance-gtInitAudio(
• NULL, // no DirectMusic interface needed
• NULL, // no DirectSound interface needed
• g_hWND, // handle to the window
• DMUS_APATH_SHARED_STEREOPLUSREVERB,//default
  audiopath
• 64, // 64 channels allocated to the audiopath
• DMUS_AUDIOF_ALL, // allow all synthesizer
  features
• NULL // default audio parameters
• )

10
Create the Music Loader

• The Music loader object loads all audio contents,
  such as MIDIs and WAVs, into segments that hold
  the auido data
• IDirectMusicLoader8 g_loader NULL
  CoCreateInstance(
• CLSID_IDirectMusicLoader,
• NULL,
• CLSCTX_INPROC,
• IID_IDirectMusicLoader8,
• (void) g_loader
• )

11
Load a Segment

• A segment is an object that encapsulates the
  sound data that is loaded from a sound file.
• We use a loader to load a segment. First we set
  the default directory to search for audio files.
• WCHAR wSearchPathMAX_PATH
• MultiByteToWideChar(CP_ACP, 0, .\\music, -1,
  wSearchPath, MAX_PATH)
• g_pLoader-gtSetSearchDirectory(
• GUID_DirectMusicAllTypes, // Types of files
  sought. wstrSearchPath, // Where to look.
• FALSE // Don't clear object data.
• )

12
Load a Segment

• Load a audio file from disk
• WCHAR filenameMAX_PATH "test.wav"
• IDirectMusicSegment8 dmusicSegment NULL
• g_loader-gtLoadObjectFromFile(
• CLSID_DirectMusicSegment, // Class identifier
• IID_IDirectMusicSegment8, // ID of
  desiredinterface.
• filename, // Filename.
• (LPVOID) dmusicSegment // Pointer that
  receivesinterface.
• )

13
Download the Band

• A band is a collection of settings for playing
  the given segment.
• The band needs to be downloaded to an audiopath
  or the performance object.
• dmusicSegment-gtDownload(dmusicPerformance)

14
Play the Segment

• Now we are ready to play the music
• Most parameters of the function can be set to 0
• g_pPerformance-gtPlaySegmentEx(
• dmusicSegment, // Segment to play.
• NULL, // Not used.
• NULL, // For transitions.
• 0, // Flags.
• 0, // Start time 0 is immediate.
• NULL, // Pointer that receives segment state.
• NULL, // Object to stop.
• NULL // Audiopath, if not default.
• )
• You can control the number of times a segment
  loops
• HRESULT IDirectMusicSegment8SetRepeats( DWORD
  dwRepeats)
• To loop forever
• dmusicSegment-gtSetRepeats( DMUS_SEG_REPEAT_INFINIT
  E )

15
Secondary Segments

• There can only one primary segment in
  DirectSound. But you can have many secondary
  segment and play them simultaneously
• g_pPerformance-gtPlaySegmentEx(
• dmusicSegment, // Segment to play.
• NULL, // Not used.
• NULL, // For transitions.
• DMUS_SEGF_SECONDARY, // Secondary segment Flag.
• 0, // Start time 0 is immediate.
• NULL, // Pointer that receives segment state.
• NULL, // Object to stop.
• NULL // Audiopath, if not default.
• )

16
Shut Down

• Stop playing a segment immediately
• dmusicPerformance-gtStopEx(dmusicSegment, 0, 0)
• Stop all sounds
• dmusicPerformance-gtStop(NULL, NULL, 0, 0)
• Unload the segment
• g_performance-gtStop(NULL, NULL, 0, 0)
• dmusicSegment-gtUnload(g_performance)
• Close down the performance
• g_performance-gtCloseDown()
• Release all allocated interfaces
• g_loader-gtRelease()
• g_performance-gtRelease()
• g_segment-gtRelease()
• Close COM
• CoUninitialize()
• Example

17
3D Sound

• 3D sound can greatly increase the realism effect
  by immersing the player.
• The coordinates system of 3D sound is similar to
  that in OpenGL except that the z axis is flipped.
• Factors affecting sound perception
• Overall loudness sounds perceived volume
  decreases as a sound source moves away.
• Interaural intensity difference A sound from
  right is slightly louder in the listeners right
  ear than the left ear.
• Interaural time difference A sound from right
  arrives at the right ear slightly earlier than
  the left ear.
• Muffling A sound from behind the listener will
  be slightly muffled compared to a sound from the
  front.

18
Using Additional Audiopaths

• Audiopaths manage the flow of sound data through
  various objects from the audio file to the final
  output.
• The simplest way to create an audiopath is to use
  InitAudio to set up a default audiopath for the
  performance. But you can create audiopath for
  more advanced features like 3D sound, stereo and
  reverb,
• IDirectMusicAudioPath8 p3DAudioPath NULL
• g_pPerformance-gtCreateStandardAudioPath(
• DMUS_APATH_DYNAMIC_3D, // AudioPath type.
• 64, // Number of performance channels.
• TRUE, // Activate now.
• p3DAudioPath // Pointer that receives
  audiopath. )

19
Playing Sound on Audiopaths

• When NULL is passed to the pAudiopath parameter
  in the IDirectMusicPerformance8PlaySegmentEx()
  function, a segment is played using the default
  audiopath.
• Pass the reference to the new audiopath to play a
  segment on this audiopath
• g_pPerformance-gtPlaySegmentEx(
• g_pSegment, // Segment to play.
• NULL, // Not used.
• NULL, // For transitions.
• DMUS_SEGF_DEFAULT, // Flags.
• 0, // Start time 0 is immediate.
• NULL, // Pointer that receives segment state.
• NULL, // Object to stop.
• p3DAudioPath // Audiopath. )

20
Retrieving Objects from Audiopaths

• you can retrieve interfaces to objects that form
  part of the audiopath and modify its properties
• Example showing how to retrieve a Sound3D buffer
  from the audiopath
• IDirectSound3DBuffer8 pDSB NULL
• p3DAudioPath-gtGetObjectInPath(
• DMUS_PCHANNEL_ALL, // Performance channel.
• DMUS_PATH_BUFFER, // Stage in the path.
• 0, // Index of buffer in chain.
• GUID_NULL, // Class of object.
• 0, // Index of object in buffer ignored.
• IID_IDirectSound3DBuffer, // GUID of desired
  interface.
• (LPVOID) pDSB // Pointer that receives
  interface.
• )

21
Setting 3D Parameters

• Usually you retrieve or set those parameters all
  at once
• DS3DBUFFER dsBufferParams
• pDSB-gtGetAllParameters(dsBufferParams)
• IDirectSound3DBuffer8SetAllParameters()
• Modify and set parameters
• // relative to the listener
• dsBufferParams.dwMode DS3DMODE_HEADRELATIVE
• ds3DBuffer-gtSetAllParameters(dsBufferParams,
  DS3D_IMMEDIATE)
• Change the position of sound
• pDSB-gtSetPosition( -0.1f, 0.0f, 0.0f,
  DS3D_IMMEDIATE )

22
3D Sound Parameters

• Minimum and Maximum Distances
• Minimum distance is the distance at which the
  sound stops increasing
• Maximum distance is the distance at which the
  sound doesnt get any quieter.
• Processing mode
• Normal sound source is positioned and oriented
  absolutely in the world
• Head-relative positioned and oriented relative
  the a listener
• Disabled 3D sound disabled.
• Position and Velocity
• Sound Cones
• Inner cone sound volume attenuates by distance
  only and not by direction
• Outer cone Between the inner and the outer sound
  cones, sound is attenuated by both distance and
  direction.
• Relative volume outside cones The maximum volume
  in the area outside the outer sound cone. Outside
  the outer cone, sound is attenuated by distance
  only

23
DirectSound 3D Listener

• DirectSound 3D listener object
• IDirectSound3DListener8 dmusicListener
• dmusic3DaudioPath-gtGetObjectInPath(
• DMU_PCHANNEL_ALL,
• DMUS_PATH_PRIMARY_BUFFER, 0,
• GUID_NULL,0,IID_IDirectSound3DListener8,
• (void )dmusicListener)
• DS3DLISTENER dsListenerParams // 3d listener
  properties
• dmusicListener-gt GetAllParameters(dsListenerParam
  s)
• // set position of listener
• dsListenerParams.vPosition.x 0.0f
• dsListenerParams.vPosition.y 0.0f
• dsListenerParams.vPosition.z 0.0f
• dmusicListener-gtSetAllParameters(dsListenerParams
  , DS3D_IMMEDIATE)
• Other properities Velocity, Front vector, Top
  vector, Doppler factor
• Example