INTRODUCTION TO MUSIC TECHNOLOGY

1
INTRODUCTION TO MUSIC TECHNOLOGY Lalya
Gaye Intelligent Products course Intelligent
System Design program IT-University of
Göteborg Fall 2003
2
SOME HISTORY

• Helmholtz resonator (1870)
• Theremin (1917)
• studios (ex IRCAM, Raymond Scott)
• synths Clavivox (1956), Moog Prodigy (1979),
  Clavia nord (now)
• samplers Akai S1100, Propellerheads Recycle
• turntable (Schaeffer, jamaican toasters, hip-hop
  Djs)
• crackle-box (STEIM)
• alternative controllers
• lap-tops

3
COMPOSER APPROACH

• Approach (cf IRCAM)
• using technology to create exactly what the
  composer has in mind
• composer tells a technician what he/she wants
• technician manipulates technology to obtain what
  composer wants
• public presented with a playback (tape-music)
• legacy
• studio music, lap-top musicians, etc
• computer used as composition tool
• screen-based interaction

4
PERFORMER APPROACH

• Approach (cf STEIM)
• using technology to perform music
• computing used as part of instrument to support
  musical expression
• real-time computer-musician interaction through
  physical interface
• focus on live performances and physicality
• legacy
• alternative (almost DIY) controllers
• putting back gesture and bodily interaction into
  music making

5
INTERACTION LOOP
input
mapping
output
sound generator
interface
1rst fb
2nd fb
6
ACOUSTIC VS. ELECTRONIC

• acoustic instruments
• sound source and interface are the same artefact
• nature of expected sound laws of mechanics
  dictate how the instrument is designed
• fixed and relatively transparent mapping
• acoustic haptic feedback
• electronic instruments
• sound soure and interface separated ? mapping,
  interaction and physical attributes are free.
• need for methods and constraints in order to
  fulfill criteria of expressiveness, transparancy,
  audio quality
• need for feedback

7
PURPOSE

• users
• experts
• amateurs
• uses
• composition
• performance
• education
• cognitive stimulation
• designing an
• instrument/interface/composition tool

8
INPUT

• movement-based (gesture/body movement)
• choreographed (dance-technology)
• instrumental
• empty-handed (ex lady-glove, the hands..)
• full-handed (ex unfoldings)
• data-based (?sonification of data)
• screen-based

9
INTERFACE

• object-oriented
• existing instruments
• augmented (ex hypercellots)
• digitalised (ex piano synth)
• interface used as controller (ex keyboard)
• everyday objects water, fabric, chemicals,
  vegetables (ex Sound Kitchen, 8Tunnel2,
  Sensordoll)
• human-based

10
INTERFACE

• environments
• interactive rooms (ex Tgarden)
• digital realm (ex Global String)
• everyday environmets (ex Sonic City)
• human-based

11
MAPPING one-to-one

• input mapping output
• X
  X
• X X
• X X
• each independent input assigned to one musical
  parameter
• simplest mapping scheme, but usually the least
  expressive

12
MAPPING one-to-many

• input mapping output
• 
  X
• X X
• X
• one input used to control more than one
  simultaneous musical parameter
• conductor model provides a macro-level
  expressivity control, but does not allow access
  to internal (micro) features

13
MAPPING many-to-one

• input mapping output
• X
• X X
• X
• many inputs coupled to produce one musical
  parameter
• requires previous experience with the system in
  order to achieve effective control
• but far more expressive than the simpler unity
  mapping

14
OUTPUT

• sound generation
• synthesised
• audio processing (incl. sampling)
• acoustic
• output interface
• headphones
• speakers
• haptics (ex Cutaneous Grooves)
• visual
• acoustic automats (ex LEMUR)

15
CONTROL

• levels of indeterminancy
• control vs. randomness (? interactive
  improvisation)
• total predeterminancy push a button ?
  deterministic output
• total undeterminancy random machines
• unexpected vs. expected input / output
• control characteristics
• continuous vs. discrete control
• implicit vs. explicit
• micro- to macro-level control spectrum to
  articulation to structure

16
FEEDBACK

• feedback
• helps articulating control
• passive vs. active
• from mono- to multi-modal (modalities audio,
  hapic, visual)
• 1rst FB from interface
• 2nd FB audio
• feedforward
• guides user by providing information about the
  internal state of the system (as opposed to
  information from output)

17
INTERACTION DURING PERFORMANCE

• performer-performer (ex Tooka)
• performer-context (ex Sonic City)
• system-audience (ex crackle-family)
• performer-system-audience (ex mobilephone
  concert)

18
SIMPLIFED SET-UP
19
CONTACT-BASED SENSORS

• isometric
• pressure
• switches
• movement sensing
• rotation pots, goniometers, joysticks
• linear movement sliders, tension sensors, pads,
  tablets
• bending

20
CONTACTLESS SENSORS

• inside-in
• emitter receiver on subject ? body-centered
• workspace in principle unlimited
• ex flex sensors, biometric sensors
• inside-out
• sensor on subject external emitter
• workspace limited if source artificial,
  unlimited if source natural
• ex accelerometers, gyroscope, compass

21
CONTACTLESS SENSORS

• outside-in
• external sensor emitter on subject
• least obtrusive
• workspace limited
• ex video tracking markers
• indirect acquisition
• deduction from audio output
• latency

22
OTHER SENSORS

• objects
• more or less same as human tracking sensors
• environment
• light
• sound
• temperature
• humidity
• electricity
• magnetism
• digital information
• ex activity on internet

23
MICRO-CONTROLLERS

• collecting sensor data and sending them to
  processor (e.g. PC) as serial data (e.g. MIDI
  signal)
• can also be used to trigger actuators (f. ex
  LED)
• usual micro-controllers
• BasicX-24
• Basic Stamp II
• PIC

24
MUSIC SOFTWARE

• using MIDI signals as control data
• usual software for interactive music
• MAX/MSP
• PD (Pure Data)

25
DIY CONTROLLER

• components
• sensors potentiometer switch
• micro-controller BasicX-24
• protocol MIDI
• software PD

26
DIY CONTROLLER

• reading sensor values with BX-24
• connect sensor to ADC pins
• power supply them with the BXs 5V DC output
  power
• (! BX power 9V)
• add SerialPort module for communicating with
  serial port
• write routine for reading voltage on pins
• download program to EPPROM

27
(No Transcript)
28
Option Explicit Dim voltIn As Byte Dim switch As
Byte Public Sub Main() voltIn 1 switch 1 Do
'potentiometer voltIn cByte(getADC(16))
'switch switch GetPin(17) Debug.Print
"voltIn" cStr(VoltIn) Debug.Print
"switch" cStr(switch) Call Sleep(0.05) Loop
End Sub
29
DIY CONTROLLER

• sending values as MIDI signal
• convert data into MIDI scale (0-127)
• create buffer
• adapt baud rate to MIDI speed
• write subroutine loop for sending MIDI
• MIDI command 144 (note on)
• velocity used as ID
• pitch used as sensor value
• download on EPPROM
• sending out serial data via MIDI adapter circuit
  and MIDI-USB adapter

30
Option Explicit Dim InputBuffer(1 To 12) As
Byte Dim OutputBuffer(1 To 10) As Byte Dim
midiCmd As Byte Dim vel As Byte Dim midiTaskVar(1
To 50) As Byte Dim voltIn As Byte Dim switch As
Byte
31
Public Sub Main() voltIn 1 switch 1 Call
openQueue(Inputbuffer, 12) Call
openQueue(Outputbuffer, 10) Call OpenCom(1, 9600,
InputBuffer, Outputbuffer) Register.ubrr
14 midiCmd 144 CallTask "midiTask",
midiTaskVar Do 'potentiometer voltIn
cByte(cSng(getADC(16)) 127.0 /
1023.0) 'switch switch GetPin(17)
Call Sleep(0.05) Loop End Sub
32
Sub midiTask () Do vel10 Call
putQueue(OutputBuffer, midiCmd, 1) Call
putQueue(OutputBuffer, voltIn, 1) Call
putQueue(OutputBuffer, vel, 1) vel20 Call
putQueue(OutputBuffer, midiCmd, 1) Call
putQueue(OutputBuffer, switch, 1) Call
putQueue(OutputBuffer, vel, 1) Call
Sleep(0.05) Loop End Sub
33
DIY CONTROLLER
sending values as MIDI signal
34
DIY CONTROLLER

• receiving MIDI data in PD
• C/pd/bin
• pd midiindev 1
• route data according to ID (vel)
• use pitch as control values

35
DIY CONTROLLER

• DIY links
• sensors
• http//www.elfa.se
• BX-24
• http//www.basicx.com
• PD
• http//www.crca.ucsd.edu/msp/software.html
• more info (ITP Physical computing)
• http//stage.itp.nyu.edu/tigoe/pcomp/index.sht
  ml

36
GENERAL LINKS

• my own link page
• http//www.viktoria.se/lalya/links.html
• NIME conference
• http//www.nime.org
• Interactive Systems and Instrument Design
• http//www.notam02.no/icma/interactivesystems/w
  g.html
• Sonic City
• http//www.viktoria.se/fal/projects/soniccity