The Use of Physiological Signals in Generative Art

1
The Use of Physiological Signals in Generative Art

• James Sheridan

2
Outline

• Physiological Signals
• Generative Art
• Similar work
• Combining the Two
• When will computers make art?

3
Goals

• Better Insight into a persons mental state
• Phase of physiological rhythms (circadian,
  hormonal, respiratory, cardiac, vascular,
  autonomic, and cellular)
• Attention levels (attended targets, shifts)
• A method for the directed control of brainwaves
  and thinking style
• Creative, attentive, relaxed etc
• Affective art
• Music and visuals that change until they are
  found interesting or useful

4
Physiological Signals

• Brainwaves (EEG)
• Neuro-feedback
• ADHD, Addiction, Creativity
• Brain Computer Interfaces (BCIs)
• Spellers, 2 words (60 bits) a minute
• Cortically coupled Computer Vision for Rapid
  Image Search

• Gaze tracking
• Gaze time
• Pupil diameter
• Movement rates
• Blink rates
• Saccades

5
(No Transcript)
6
Evolutionary Computing

• Rule 110
• Capable of universal computation
• Turning complete

'a aC C', 'a bC D', 'b aD C', 'b bD D', 'cacAAA',
'dacBBB', 'cbcBBB', 'dbcBBB', 'cadAAA', 'dadBBB',
'cbdBBB', 'dbdAAA'
(Wolfram 2002, p. 1115 Cook 2004)
7
Generative Art

• Are these Art?
• Evolving music for a computer game
• Fractals
• Sonification

8
(No Transcript)
9
Similar work

• Memetic theory
• Meme an information pattern, held in an
  individual's  memory, which is capable of being
  copied to another individual's memory. 
• Memetics the theoretical and empirical science
  that studies the replication, spread and 
  evolution of memes
• Reunion Chess game (John Cage, Marcel Duchamp
  and Teeny Duchamp, late 1960s)
• The Musicator produces music by associating the
  moves of the game (connect 4) with musical forms.
• The synthetic modelling of language origins,
  Evolution of Communication (Steels, L 1997)
• Neuro-feedback for Musicians

10
(No Transcript)
11
Emergence

• Chaosynth uses Cellular Automata (CA) to control
  a granular synthesis algorithm
• (Eduardo R Miranda 2003)
• CAMUS uses two simultaneous cellular automata
  (CA) to generate musical forms the Game of Life
  and Demon Cyclic Space.
• (Eduardo R Miranda 2003)

12
Why use this Approach?

• Current Neuro-feedback problems
• Uni-modal
• Static mappings
• Boring
• Explore brainwave signal space
• We dont understand the neural coding of
  brainwaves
• Explore new methods of composition
• Blends the lines of who is making the piece, the
  designer, the rules or the listener

13
Brain Features

• Herd mentality
• Mirror neurons
• Self organisation
• Imitation
• Hypothesis A person is more likely to respond
  to Neuro-feedback if another similar entity is
  involved

14
How to alterbrainwaves?

• Chanting Religious, Healing ceremonies, Tribal,
  Sporting, Warrior
• Mantras
• Meditation
• Audio weapons - non-lethal weapons for use in
  crowd control and coercive interrogation
• Music
• Tempo/BPM (Peretz)
• Expectancies
• Duration, Rhythm, Structure
• Timbre, Pitch contours

15
Resonance

• Ancient Acoustics
• Wayland's Smithy, Chun Quoit, and Cairn Euny, all
  in the U.K. Newgrange, and Cairns L and I,
  Carbane West, all in Ireland.
• Pyramids
• Mayan temples

(Jahn, Robert G., et al "Acoustical Resonances
of Assorted Ancient Structures," Technical Report
PEAR 95002, Princeton University, March 1995.
Devereux, Paul, et al "Acoustical Properties of
Ancient Ceremonial Sites," Journal of Scientific
Exploration, 9438, 1995.)
16
Binaural beats

• Result from the interaction of two different
  auditory impulses, originating in opposite ears,
  below 1000 Hz and which differ in frequency
  between one and 30 Hz (Oster, 1973)
• Frequency Following Response originates from the
  inferior colliculus (Smith, Marsh, Brown, 1975)
  
• Originate in the superior olivary nucleus of each
  hemisphere

17
Wellness and Mental Fitness
18
Architecture
Electroencephalograph
Gaze tracking system
Position Trackers
Data Synchronisation, Pre Processing and Serving
Logging Playback
Neuron Model
Sonification, Visualisation system events
Generative control
Sound Synthesis
VR Display System
19
(No Transcript)
20

Testing it works

• How can one evaluate a system for musical
  composition objectively?
• How can you be sure neuro-feedback is eliciting a
  response?

21
Testing it works
Attention
Feedback
22
When will computers make art?

• When they elicit an emotional response?
• When they can learn new things on their own?
• When they can consciously reflect upon their work

23
The I went to quick slide

• It is probably true quite generally that in the
  history of human thinking the most fruitful
  developments frequently take place at those
  points where two different lines of thought
  meet.  These lines may have their roots in quite
  different parts of human nature, in different
  times or different cultural environments or
  different religious traditions  hence if they
  actually meet, that is, if they are at least so
  much related to each other that a real
  interaction can take place, then one may hope
  that new and interesting developments may follow
• - Werner Heisenberg, founder of quantum
  mechanics