Music and Emotion

1
Music and Emotion

• Perfecto Herrera
• Music Perception and Cognition

2
(No Transcript)
3
Music and Emotion

• Does music convey specific emotions? How do we
  experience that?
• Does music really evoke emotions?
• If music evokes emotions, why do we emotionally
  react to it?
• Is it possible to turn a emotionless performance
  into another one that conveys some emotion?
• Do we really want to know the answer to the
  previous questions or are we going to destroy the
  magic of music?
• Is it possible a music-based emotion engineering?

4
What is an emotion (I)?

• "everyone knows what an emotion is, until asked
  to give a definition"
• (Fehr Russell, 1984, p.464)
• Emotion in both an everyday concept and a
  scientific construct
• Involves both an implicit and an explicit body of
  knowledge
• Emotion as opposed to reason (Descartes error,
  Damasio 1994)
• Emotions have survival value, they are not bells
  and whistles of behavior

5
Affective phenomena (from Scherer, 2000)
6
What is an emotion (II)?

• Emotion is a complex set of interactions among
  subjective and objective factors, mediated by
  neural/hormonal systems, which can
• give rise to affective experiences such as
  feelings of arousal (activation at different
  behavior levels), pleasure/displeasure
• generate cognitive processes (e.g. increasing
  attention, appraisals, labeling processes)
• activate widespread physiological adjustments
  (e.g., increasing heart-rate, sweating, crying)
• lead to behavior that is often, but not always,
  expressive, goal-directed, and adaptive (e.g.,
  running away, reiterating exposure)

7
The limbic system
Prefrontal cortex modulation of the action plans
Auditory cortex
Nuclueus accumbens reward
Hippocampus episodic memory links to past
experiences
Amygdala survival value/behavior good, bad,
fly, chase, mate
Hypothalamus-hypophysis hormonal release
8
The amygdala

• Connecting (associating) external stimuli with
  internal values
• Not active when parents heard children crying
  that were not their own
• More active when listening emotional music with
  eyes closed (eyes-closed increasing danger
  potential -gt more alert)
• A modulator to ensure that emotional responses
  are appropriate to the external stimuli and to
  the social context
• Active role in different emotion-related tasks
  fear conditioning, contextual conditioning,
  recognition of fearful facial expressions, and
  emotion-guided decision making

9
Primary and secondary emotions

• Primary (limbic, fast circuit thalamus-amygdala)
  versus secondary (cortical, slow circuit
  thalamus-cortex-amygdala) emotions
• Primary emotions require external input
  secondary emotions can be generated by internal
  working (thoughts)
• Somatic marker hypothesis (Damasio, 1994) when
  a thought generates a frontal cortex emotional
  response, the thought is "marked" by the physical
  emotion.
• Likewise, cognitive processing of the thought is
  affected by the neurotransmitters evoked by the
  emotion. Further processing of the thought will
  be inhibited by negative emotions, and
  facilitated by positive emotions.
• The frontal cortices send signals to the limbic
  system to generate an emotional response. These
  signals may be full-fledged emotions, or may only
  produce part of the response spectrum, such as
  neurotransmitter release.

10
Music and emotion

• Gabrielsson (1995)
• There is an isomorphism between the structure of
  music and the structure of feelings
• Wagner
• Music begins when language ends
• Music is the language of passion
• Stravinsky
• Music by its nature, is essentially powerless to
  express anything at all expression has never
  been an inherent property of music

11
Music as a way of conveying emotions

• Langner (1942)
• Music represents the dynamic form of emotional
  life, not specific emotions
• Music is a tonal analogue to emotive life
• Music reveals the nature of feelings that
  language cannot approach

12
Music as a way of conveying emotions
13
Methodological issues

• How to measure emotion in musical-listening
  contexts?
• Word-lists, ratings and self-reports
• Facial expression (micro changes detectable with
  electromyography)
• Behavioral changes (crying, avoidance)
• Physiological parameters (breath and heart rate,
  skin conductance changes)
• fMRI and ERP
• Between-subjects variability
• Within-subject reliability
• Reactivity (what we try to measure disappears
  or fades because of the laboratory conditions)
• Cultural issues (emotions are expressed
  differently in different cultures equivalent
  words in 2 languages cannot be emotionally
  equivalent)

14
Music-induced vs. music-expressed emotions

• Induced Music affects the emotional state of the
  listener
• Expressed Music is categorized as an emotional
  content conveyor

15
Why does music convey emotion?

• Hearing resemblance between the music and the
  natural expression of the emotion (similarity to
  speech).
• One example can be anger where the loudness and
  the spectral dissonance (derived from frequency
  ratios and harmonic coincidence based on
  psychoacoustic tests) are two components we can
  find in both an angry voice and music.
• Accumulated connotations a certain musical
  phenomena acquire in a culture we learn in our
  culture which musical cues correspond to which
  feeling.
• One example can be brass instrumentation and slow
  tempo meaning solemnity
• Kivy (1989)

16
Why does music induce emotions?

• Musical expectation we continuously generate and
  hold expectations, that the music
  as-it-develops-in-time confirms or denies (Meyer,
  1956)
• Arousal music acts as a source of cognitive
  activation by means of its speed, complexity,
  beauty or familiarity. Arousal might follow the
  amount of complexity we find in stimuli (we tend
  to prefer and enjoy moderate levels of
  complexity, beauty, etc. (U-shaped function by
  Berlyne, 1971)
• Mood contagion there are similarities between
  emotional language features and musical features
  (e.g., pitch contour, prosody) finding them in
  music makes the learned emotion to be developed.
  Contagiousness can also be developed by observing
  audiences happy music induces more smiles, sad
  music induces frowning, these behaviors can be
  unconsciously imitated, and then the emotion can
  be activated (inverse process of emotion
  induction)

17
Why does music induce emotions?

• Associations music-activated emotions reflect
  personal and specific associations that can be
  arbitrary with respect to the music content.
  Non-musical features can affect the emotional
  reaction (e.g. they are playing our song).
  Association is learnt by conditioning, not
  needing our conscious will. Recall from episodic
  memory can be favored by music when subjects have
  memory impairments (Sacks, 2007) .
• Music imagery We do not need the acoustic
  stimulus of music. Our musical imagination, by
  means of association can activate mental images
  related to emotionally-charged activities,
  persons, places.

18
Basic emotions

• They are easily distinguished in all the
  nonverbal modalities (see photo)
• They are linked to survival issues danger,
  competition, loss cooperation

• Complex emotions (anger, contentment, curiosity,
  disgust, fear, happiness, jealousy, love, pride,
  sadness, shame, and tenderness) are more
  difficult to communicate by musical means and to
  detect by visual inspection of fMRI

19
Musical expectation emotions as a function of
monitoring match and mismatch

• Most compositional systems (e.g., tonal systems)
  provide a set of dimensions that establish
  psychological distance from 'home' or 'stability
  point'
• Proximity or approach to this resting point
  involves reduction of tension
• Distance can be measured on a number of
  dimensions such as rhythm and meter (strong
  beats are stable, weak beats and syncopations are
  unstable) and tonality (the tonic is stable,
  non-diatonic notes are unstable)
• A moderate amount of surprise and fulfillment of
  expectations (tension-relaxation) is preferred
  (again the inverted-U curve proposed by Berlyne)

20
Musical expectation emotions as a function of
monitoring match and mismatch

• If musical expectation is really the key to
  emotional intensity, how is it that we can feel
  emotions to music we are highly familiar with?

21
Musical expectation emotions as a function of
monitoring match and mismatch

• Many of the violations of expectations may occur
  on a subconscious level
• Even when the musical 'narrative' is familiar to
  us, we may still be able to enjoy it. We can
  appreciate the twists and turns (like re-watching
  a great movie)
• Iconic and associative sources of emotion, such
  as emotional contagion and memories, may remain
  much the same throughout repeated listening to
  the same piece of music
• Familiarity with an object itself might increase
  our liking of that object up to a certain point
• It is possible that some effects of music
  processing is executed by a processor whose
  responses are 'hard-wired' in regard to certain
  perceptual primitives

22
Categorical representation
Adjectives and clusters, inspiredin Hevner
(1936)
23

• Hevner, 1935
• Stimulus
• 10 short musical excerpts by Schumann, Bach,
  Beethoven, Gluck
• 2 versions major / minor
• Similar performances
• Subjects / Procedure
• Students, 4 groups
• Each heard 5 major 5 minor
• Replicated by Terwogt van Grinsven (1991)
• Adults more reliable than children or youngsters
• Anger and fear tend to be confused

24
Dimensional representation
Russells (1980) circumplex model 2-D space
spanned by valence and arousal emotion
categories are defined by different combinations
of them
25
Dimensional representation with determining
features
From Juslin (2001)
Positive Valence
TENDERNESS slow mean tempo (Ga96) slow tone
attacks (Ga96) low sound level (Ga96) small sound
level variability (Ga96) legato articulation
(Ga96) soft timbre (Ga96) large timing variations
(Ga96) accents on stable notes (Li99) soft
duration contrasts (Ga96) final ritardando (Ga96)
HAPPINESS fast mean tempo (Ga95) small tempo
variability (Ju99) staccato articulation
(Ju99) large articulation variability (Ju99) high
sound level (Ju00) little sound level variability
(Ju99) bright timbre (Ga96) fast tone attacks
(Ko76) small timing variations (Ju/La00) sharp
duration contrasts (Ga96) rising micro-intonation
(Ra96)
High Activity
Low Activity
ANGER high sound level (Ju00) sharp timbre
(Ju00) spectral noise (Ga96) fast mean tempo
(Ju97a) small tempo variability (Ju99) staccato
articulation (Ju99) abrupt tone attacks
(Ko76) sharp duration contrasts (Ga96) accents on
unstable notes (Li99) large vibrato extent
(Oh96b) no ritardando (Ga96)
FEAR staccato articulation (Ju97a) very low
sound level (Ju00) large sound level variability
(Ju99) fast mean tempo (Ju99) large tempo
variability (Ju99) large timing variations
(Ga96) soft spectrum (Ju00) sharp
micro-intonation (Oh96b) fast, shallow, irregular
vibrato (Ko00)
SADNESS slow mean tempo (Ga95) legato
articulation (Ju97a) small articulation
variability (Ju99) low sound level (Ju00) dull
timbre (Ju00) large timing variations (Ga96) soft
duration contrasts (Ga96) slow tone attacks
(Ko76) flat micro-intonation (Ba97) slow vibrato
(Ko00) final ritardando (Ga96)
Negative Valence
26
Factors used to express emotions
27
Factors used to express emotions

• Laurier et al. (simple rules using
  acoustic/musical descriptors capable to predict
  with correctnessgt80 the expressed emotions in
  MP3 collections)
• dissonance_dvar lt 0.002717 aggressive
• dissonance_dvar gt 0.002717
• AND spectral_complexity_mean lt 13.557088
  not_aggressive
• AND spectral_complexity_mean gt 13.557088 
  aggressive
• Lyrics content analysis added lt5 of additional
  labelling power

28
Extended lens model (Brunswik Juslin
Lindström)
29
Synthesis of Emotion
An example of the analysis by synthesis
strategy different renditions of the same piece
are synthesized by changing musical parameters,
then we study the effect of the parameter-tuning
on the perceived emotions
30
Emotion in time
31
Social-emotional aspects

• Massive surveys on music preference and
  personality features
• Relationships between personality traits and
  music preference
• Relationships between social groups and music
  preference
• Music as the preferred way to present oneself
  to others, specially for teenagers and youngsters
• Heartgraves North
• http//www.peopleintomusic.com/
• Rentfrow Gosling Short Test on Musical
  Preferences (STOMP)
• http//www.outofservice.com/music-personality-test
  /
• http//www.signalpatterns.com/

32

• P.N. Juslin J.A. Sloboda (Eds)(2001). Music and
  Emotion. Oxford Oxford University Press
• Jourdain. R. (1997). Music, the Brain, and
  Ecstasy How Music Captures Our Imagination. New
  York William Morrow and Company
• Course on Music and Emotion by David Huron
  http//csml.som.ohio-state-edu/Music829D.html