LIVE AUDIO

1
LIVE AUDIO VISUALS WORKSHOPPART I

• STEVE GIBSON
• ASSOCIATE PROFESSOR, DIGITAL MEDIA
• UNIVERSITY OF VICTORIA
• VICTORIA, BC CANADA

2
INTRO TO LIVE AUDIO VISUALS

• Software Introduction Ableton Live
• http//www.ableton.com/live
• Ableton Live is a tool for composing, processing
  and performing audio/music with enhanced
  possibilities for real-time interaction.
• Live input from various sources (microphones,
  music instruments) can be processed in real-time
  with audio effects.
• Audio effect parameters can also be processed in
  real-time using MIDI controls from various
  sources (control surfaces, keyboards, etc.).
• Educational Pricing Full version EUR 209/USD 249
• Demo version also available for download.

3
ABLETON LIVE INTERFACE
4
INTRO TO LIVE AUDIO VISUALS

• Software Introduction Modul8
• http//www.garagecube.com/modul8/index.php
• Modul8 is a tool for processing video data in
  real-time.
• Live camera input can be processed live with
  effects and mixed with pre-prepared video content
  (including text-based video).
• Video effects on the camera and pre-prepared
  video content can be altered in real-time using
  MIDI control.
• Educational Pricing EUR 199 / 5 site license EUR
  472.
• Demo version also available for download.

5
MODUL8 INTERFACE
6
INTRO TO LIVE AUDIO VISUALS

• Approaches to Live Audio and Visuals
• Installation Art
• In installation art live audio and visuals are
  usually triggered by external inputs monitored by
  some sort of motion-tracking, camera-tracking or
  sensor device.
• Most systems are custom-built for specific
  operations.
• Many simple sensor systems are pre-made to order
  and feature MIDI output for easy access to
  control information.
• Example
• MIDISense - http//www.ladyada.net/make/midisense/

7
INTRO TO LIVE AUDIO VISUALS

• Approaches to Live Audio and Visuals
• Live Audio-Visual Performance
• In live audio-visual performance software such as
  Max-MSP or Ableton Live is used in conjunction
  with live video software such as Jitter (with
  Max) or Modul8.
• Digital Signal Processing (DSP) is used in video
  software to match video changes with audio input.
• More complex systems such Soundium (created by
  Stefan and his Corebounce team) also allow for
  much more complex audio processing and video
  response
• Example
• Max/Jitter - http//www.cycling74.com/products/max
  5

8
INTRO TO LIVE AUDIO VISUALS

• Approaches to Live Audio and Visuals
• Live Electronic Literary Work
• In live literary work with audio-visual support
  there is not as clearly defined an approach as in
  installation art.
• Most live literary work uses live vocal
  performance enhanced by simple live graphics
  (i.e. Flash) with pre-prepared audio.
• More complex literary works feature more
  customized interfaces.
• Example
• When Ghosts Will Die - http//www.telebody.ws/Ghos
  ts/

9
INTRO TO LIVE AUDIO VISUALS

• Examples of Projects With Live Audio and Visuals
• Camera Pixelizer by Ross Macaulay Sean Gordon
• This piece by two Digital Media undergrads at
  University of Victoria uses a live wireless
  camera to capture footage which is then played
  back on a TV with a specially-created pixelizer.
• Live mic audio is processed with audio effects in
  real-time by Ableton Live.

10
CAMERA PIXELIZER
11
INTRO TO LIVE AUDIO VISUALS

• Examples of Projects With Live Audio and Visuals
• Devils Rope Journal by Randy Adams, babel
  Christine Wilks
• This piece involved several media artists from
  around the world who agreed to work together
  using open source blogging software to present
  and remix audio pieces, Flash animations, digital
  images, visual poetry and texts.
• The live context text was spoken in tandem with
  live Flash animations and pre-recorded audio.

12
DEVILS ROPE JOURNAL
13
INTRO TO LIVE AUDIO VISUALS

• Examples of Projects With Live Audio and Visuals
• Exploding, Plastic Inevitable Redux by Steve
  Gibson Stefan Müller Arisona with guests
• This piece involves multi-screen projections and
  live audio interacting to create a hallucigenic
  effect.
• Live video and audio software is used to create
  interactions between the visual and auditory
  realms.
• Collective improvisation in both realms is
  achieved through the use of complex real-time
  interaction in Ableton Live, Soundium and Modul8.

14
EXPLODING, PLASTIC INEVITABLE
15
MIDI SPECIFICATIONS

• MIDI Specification Basics
• MIDI Musical Instrument Digital Interface
• Example a MIDI setup with In and Out
  connections.

16
MIDI SPECIFICATIONS

• MIDI Specification Basics
• MIDI is like sheet music for computers. It
  allows you to record something from one keyboard
  into a computer and have the computer replay that
  music on the manner of a player piano back to the
  original keyboard or another MIDI device.
• MIDI also allows you to control audio and video
  parameters in real-time with very little memory
  or CPU usage.
• MIDI interface - a device that connects to your
  computer (via USB or Firewire ports) and allows
  to connect MIDI devices IN to your computer and
  routes MIDI data OUT to your MIDI devices.

17
MIDI SPECIFICATIONS

• MIDI Specification Basics
• MIDI Out - transmits MIDI information (notes,
  volume data, pan data, etc.) to wherever it is
  connected to.
• MIDI In - receives MIDI data from another device.
• MIDI channels - there are sixteen channels
  allowed by MIDI, which means you can have one
  sound on one channel (piano) and another on a
  second channel (drums) and the computer will tell
  the difference between the two.
• There are usually two ports in a MIDI setup so
  this allows for 2 x 16 channels.

18
MIDI SPECIFICATIONS

• MIDI Specification Basics
• When you record from a keyboard into a computer
  the data is passed out of the MIDI OUT of a
  synthesizer to the MIDI IN of your MIDI interface
  and then into your MIDI software where it is
  stored as raw textual data, NOT sound.
• You can also use MIDI data from synthesizers and
  control surfaces to control audio and video
  parameters in real-time.

19
MIDI SPECIFICATIONS

• MIDI Specification Basics
• Note On - occurs when you hit a key. Note values
  are between 0 or C-2 (lowest note) and 127 or G8
  (highest note).
• Note Off - occurs when you release a key.
• Velocity - Measures the hardness you hit a key
  with and is usually mapped to volume. Measured
  between 1 (soft) and 127 (hard).
• Pitch bend - Allows you you bend the frequency of
  a note via a wheel.
• Control change - Allow you to control a number of
  musical parameters, such as volume, pan or
  modulation.
• Tempo changes - You can change the speed at which
  a sequence plays. Measured in beats-per-minute
  (bpm).

20
SOFTWARE DETAIL ABLETON LIVE

• Ableton Live
• Live has two views Session View and Arrangement
  View.
• We will concentrate on the Session View.
• The internal computer mic can be set as default
  input along with internal audio audio.
• If you use an audio interface its ins and outs
  can also be used.

21
ABLETON LIVE AUDIO PREFS
22
ABLETON LIVE TRACKS
23
ABLETON LIVE EFFECTS
24
ABLETON LIVE MIDI MAPPING
25
WORKING WITH AUDIO IN ABLETON

• Live can import the following file types aiff,
  wav, mp3.
• The most sophisticated feature of Live is its
  ability to warp imported sound files to a
  common tempo.
• This allows you to mix and match imported audio
  files from various sources.

26
ADD AUDIO TRACKS
27
ABLETON AUDIO CLIP FEATURES

• Live can deal with audio clips in various ways.

28
ABLETON AUDIO CLIP FEATURES

• The audio clip can be zoomed in.

29
ABLETON AUDIO CLIP FEATURES

• The start point can be corrected to match the
  first beat. This is especially important for
  rhythmically-oriented clips like drum beats.

30
ABLETON AUDIO CLIP FEATURES

• Portions of the clip can be looped and audio
  envelopes (such as volume) can be drawn in.

31
CROSSFADE TWO TRACKS
32
CROSSFADE TWO TRACKS
33
SOFTWARE DETAIL MODUL8

• MODUL8
• Modul8 allows the user to import video files and
  mix and match them with live camera feeds as well
  as text.
• Modul8 will read various types of digital video
  files but for best performance you should use the
  following format
• QuickTime movie, 15-30 fps, Photo - JPEG
  Compressor
• 640x480 OR 360x240 frame size (depending on your
  computers processing power and video card clock
  speed)
• It is VERY important to use the Photo - JPEG
  Compressor when exporting from your video editing
  software, particularly if you are using a lot of
  movies in Modul8.

34
SOFTWARE DETAIL MODUL8

• MODUL8
• The Photo - JPEG compressor provides very
  efficient de-compression and therefore you will
  notice very little lag when triggering your
  movies in Modul8.
• It is important to keep your ALL of your imported
  files in one frame size (i.e. 640x480 OR 360x240)
  for each project.
• It is also important to store/import your video
  files from one location as Modul8 links to video
  files rather than stores them in the file.

35
MODUL8 PREFERENCES

• To access Modul8 Preferences select the Modul8
  Menu Preferences.

36
MODUL8 PREFERENCES
37
MODUL8 PREFERENCES
38
MODUL8 INTERFACE
39
MODUL8 VIDEO INPUT
40
MODUL8 VIDEO FX
41
MODUL8 VIDEO CLIPS

• To Import a clip into Modul8 choose Command-I.
• To select a video for playback in the video
  preview window simply click on its thumbnail in
  the bank.
• For info about the clip or to move it around to
  another slot click the small orange i button (see
  next slide).

42
MODUL8 VIDEO CLIPS
43
MODUL8 VIDEO CLIPS
44
MODUL8 MIDI MAPPING
45
MODUL8 MIDI MAPPING

• To map MIDI controls to only one layer use Focus
  Mode. For multiple layers use Position Mode.

46
MODUL8 MODULES

• Modul8 can also use a number of external Modules
  to control video parameters.

47
MODUL8 MODULES
48
MODUL8 TEXT MODULE