Framework for Live Algorithms

1
Framework for Live Algorithms

• Tim Blackwell
• Michael Young
• Goldsmiths College, London
• www.timblackwell.com
• Organised Sound 9(2) 123136

2

• The context
• Consider (for now, because its easier)
  improvised performance
• i.e. free of compositional directives such as
  harmony, rhythm, form
• Conventionally such improvisations are
  explorations of timbre and texture

3

• The context
• Consider (for now, because its easier)
  improvised performance
• i.e. free of compositional directives such as
  harmony, rhythm, form
• Conventionally such improvisations are
  explorations of timbre and texture

4

• What is a Live Algorithm?

5

• A Live Algorithm is
• Interactive
• Autonomous
• Ideas generator
• Idiosyncratic
• Comprehensible

6

• Interactive
• Sympathetic, supportive, makes appropriate
  contributions, tacets
• Autonomous
• Not merely automatic, mechanical and
  predictable. Must be comprehensibly interactive
  and capable of novelty
• Ideas machine
• Contradictory, individualistic
• source of novelty and leadership

7

• Idiosyncratic
• Contributions limited by instrument and
  experience but can be unusual, individualistic
• Comprehensible
• Suitable for collaboration, at least by humans.
  Not opaque, but not too transparent either
• And, importantly

8

• Closure
• knows when to stop!

9

• Principles
• Expectation of generating form might not be
  necessary
• Local events can be structuring
• For example, spatio-temporal self-organisation
  depends only on local interactions of a certain
  complexity and on positive and negative feedback.
• Here, the space is (interpreted as) a
  musical/sonic parameter space at some level.
• Parameterisations possible at micro (sample,
  grain), mini (note) and meso (phrase) levels. The
  problem of emergence might necessitate a need a
  multi-level system

10

• Knowledge of music rules might not be necessary
• LA need not be aware of what it is doing
• In the Y model, LAs and humans interact with
  meaningless sounds, populating an inert
  environment.
• This builds on a former XY model, which
  expresses the paradox of interaction
• (In our nature-inspired systems, The sounds are
  organised by stigmergy)

11

• Knowledge of music rules might not be necessary
• LA need not be aware of what it is doing
• Desirability of an Interactive Model and a
  Conceptual Architecture
• In the Y model, LAs and humans interact with
  meaningless sounds, populating an inert
  environment.
• This builds on a former XY model, which
  expresses the paradox of interaction
• (In our nature-inspired systems, The sounds are
  organised by stigmergy)

12
The XY model

13
The Y model

14

• Conceptual (and Actual?) Architecture
• P Listening, analysing
• Typically thins degrees of freedom from real
  time
• YIN -gt p(t) -gt pi
• F(p) Ideas engine
• Patterning in a hidden space H
  generative/algorithmic/iterative
• xi -gt xi1
• Q Interpretation, Playing, synthesizing
• Typically expands d.o.f. into real time
• xi1 -gt q(t) -gt YOUT

15
PfQ Architecture
16

• Analysis
• is typically projection from one level to
  another, higher level. E.g. samples Y to event
  parameters p
• P Y(t) -gt p(t) -gt pi
• We observe that this projection is determined by
  cultural, personal, genre-specific and even
  political forces
• P might be a map into H and hence hidden state x
  is a possible parameterisation of the sonic
  environment.

17

• Interpretation
• is the process of relating internal states to
  event parameterisations, and eventually to sound
• Q xi1 -gt q(t) -gt YOUT
• A technical complication surrounds the
  relationship between real and algorithmic time
• It is unlikely that both will flow at the same
  rate the computational update time xi -gt xi1
  might not correspond with the desired time
  interval between events Y(t), Y(tDt), ...
• Internal states might be sampled at a given rate
  irrespective of iterative time or timing
  information could even derive from x itself
• Information might be derived from averages over
  a population x1, x2, x3, ...
• All these complications and possibilities are
  represented by a single interpretative function Q

18

• Ideas generator
• F is parameterised by p and determines state
  flow of hidden variables x
• F(p) xi -gt xi1
• xi1 determined F, p(t) and xi
• F need not derive from any musical concern, and
  this may be advantageous
• F only concerns patterning and structure

19

• Autonomous
• 1. Dx ? 0 even if Dp 0
• 2. Dx 0 even if Dp ? 0
• 1. This is usual, expresses state flow in a
  dynamical system
• 2. Harder to achieve, but could arrange for Q( x
  Î?x-dx, xdx ) q.
• Alternatively, include dissipation and
  re-energising so that Dx 0 between injections
  of energy (determined by S p?)

20

• Idiosyncratic
• Q and P concern the mappings to and from sound
  herein lies the machine's idiosyncrasies and
  characteristics
• Can be quite limited, since interactivity and
  autonomy are the major prerequisites
• Potentially P, Q may cross levels

21

22

• Interactive
• autonomy modification of system state
• Analysis parameters p ensure interaction i.e. Y
  affects, but does not fully determine, x
• For example, p might be an attractor in H. In a
  dynamic system, x orbits p, but precise
  trajectory depends on initial conditions
• Alternatively, hidden variables x can be
  regarded as parameters which select mappings Fx
  p -gt q
• The collaborative interface is the asynchronous
  map
• Q Fx P Mx
• YOUT(tDt) Mx YIN(t)

23

• Comprehensible
• Transparency gt Comprehensibilty gt Opacity
• Very transparent if p Î H, Q P-1 and dim(H)
  is small enough.
• Easy to establish correlations between Yhuman
  (incoming sound, deposited in the environment by
  a human) and YLA (outgoing sound deposited by the
  LA). Might become too predictable unless enhanced
  by some stochastic adjustment (a degenerate
  solution)
• Very opaque if QFP is very complex and dim(H) is
  big.
• Very hard to establish correlations or any
  measure of causal relations between Yhuman and
  YLA. Too unpredictable and therefore not
  collaborative

24

• Closure
• Raises issues of knowledge of form
• Could argue that LA should not be deprived of
  knowing the modus operandi of the performance
• In humans, closure is influenced by time
  constraints, visual cues and an increasing
  capacity to interpret gestures as possible sonic
  cadences
• However, in an LA closure might possibly emerge
  from lower level dynamics e.g. self-organised
  criticality

25

• Other desirable properties
• Memory
• Dynamical states do not conventionally posses
  this
• Short term (repetitions, anticipations based on
  recognition..)
• gtgtgt attractor persistence, pheromone trails
• E.g. Swarm Techtiles, Ant Colony simulations
• Long term (draws on previous musical
  engagements) gtgtgt current states could have
  memories of previous states
• E.g. Particle Swarms (particles have a memory
  and participate in social networks)
• AND the sequence of conditions that caused
  these states (a contextual memory)
• E.g. ??

26

• Links with existing research fields
• P Y -gt p
• Real-time music analysis and informatics
• Q x -gt Y
• Real-time audio synthesis
• F
• Generative Music (Neural, Cellular Autonoma,
  Genetic Algorithms, Swarms

27

• Links with existing paradigms
• Live Electronics
• ideas engine F replaced by human volition. State
  flow is adjustment of controls x
• Live Coding
• F I and Q is adjusted (in software) manually
• Generative/algorithmic music
• Set Dp 0 (turn P off)
• Y(t) F(xi) FN( x0 )
• The system is self-contained - the composer
  chooses F and the initial condition, x(0)

28

• Reactive Systems
• In a reactive system, YIN will necessarily
  trigger certain transitions xi -gt xi1
• Mx QFP express a causal and necessary chain
  a rule-based system
• It is conjectured that a reactive system might
  not be sufficient for a Live Algorithm because it
  is not sufficiently interactive
• This touches on various issues in cognitive
  science and machine intelligence
• Possibly, a large enough rule set might do the
  job, and might also be indistinguishable from a
  discrete dynamical system (i.e. an algorithmic
  model of a continuous DS)
• It is hard to see how we might achieve this
  complexity without drawing on inspiration from
  dynamical, and other natural, systems

29

• (Our) Experience
• Swarm Music
• Very transparent, P and Q operate at the note
  level
• Swarm Granulator
• Less transparent, functions largely at the grain
  level, although some level-crossing

30

• Swarm Tech-tiles
• P actually expands Y into a 2D landscape
• F integrates ALife and optimisation
• Q is a rendering of parts of the landscape
  visited by x
• Argrophyllax
• P FFT analysis
• f stochastic function
• H Fourier space
• Q coefficients of inverse FFT transform Q

31

• Evaluation
• Irrelevance of Turing Test? TT is more relevant
  for the performers than the listeners. A metric
  do the performers find the LA to be stucturing?
• Applicability
• The attributes of a LA should be useful in other
  musical contexts
• E.g. intelligent effects pedal, synth plug-ins,
  accompaniment programs, genre improvisation