Introduction to Cognitive Science

1
16th International Summer School in Cognitive
Science, New Bulgarian University, Sofia,
Bulgaria, 2009
The Embodied Cognition Literature
contd
Michael J. Spivey Department of Cognitive
Science University of California, Merced
2
Embodied Cognition
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
Visual Properties Influence Concepts
Motor Properties Influence Vision
Language Influences Visual Properties

Language and Motor Influence Each Other

Your Entire Body Participates in Thought

3
Embodied Cognition
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
Scheerer Lyons (1957)
4
Embodied Cognition
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
5
Embodied Cognition
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
Visual Properties Influence Concepts
Motor Properties Influence Vision
Language Influences Visual Properties

Language and Motor Influence Each Other

Your Entire Body Participates in Thought

6
Embodied Cognition
Summer School in Cognitive Science
Barsalou, Solomon, Wu (1999)
List the features of a watermelon
List the features of half a watermelon
Large Green Sweet
Red Sweet Seeds
7
Embodied Cognition
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
8
Embodied Cognition
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
Visual Properties Influence Concepts
Motor Properties Influence Vision
Language Influences Visual Properties

Language and Motor Influence Each Other

Your Entire Body Participates in Thought

9
Vision Coextensive with Action
Summer School in Cognitive Science
Stimulus-Response Compatibility
Tucker Ellis (1998)
10
Vision Coextensive with Action
Summer School in Cognitive Science
Stimulus-Response Compatibility
Tucker Ellis (1998)
11
Cognition Coextensive with Action
Summer School in Cognitive Science
Stimulus-Response Compatibility
Tucker Ellis (1998)
12
Cognition Coextensive with Action
Summer School in Cognitive Science
Stimulus-Response Compatibility
Tucker Ellis (1998)
13
Cognition Coextensive with Action
Summer School in Cognitive Science
The Importance of Being Somatosensory Input
Bosbach, Cole, Prinz, Knoblich (2005)
14
Embodied Cognition
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
15
Embodied Cognition
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
Visual Properties Influence Concepts
Motor Properties Influence Vision
Language Influences Visual Properties

Language and Motor Influence Each Other

Your Entire Body Participates in Thought

16
Mental Simulations
Language Coextensive with Imagery
Summer School in Cognitive Science
Stanfield Zwaan (2001)
John put the pencil in the cup. John put the
pencil in the drawer.
17
Mental Simulations
Language Coextensive with Imagery
Summer School in Cognitive Science
Zwaan, Stanfield Yaxley (2001)
The ranger saw the eagle in the sky. The ranger
saw the eagle in the nest.
18
Mental Simulations
Language Coextensive with Imagery
Summer School in Cognitive Science
Zwaan, Madden, Yaxley, Aveyard (2004)
The pitcher hurled the softball at you. You
hurled the softball at the pitcher.
19
Mental Simulations
Language Coextensive with Imagery
Summer School in Cognitive Science
Meteyard, Bahrami, Vigliocco (2007)
Downward
bomb, drop, fall
Control
cook, sew, write
Upward
climb, rise, fly
20
Mental Simulations
Language Coextensive with Imagery
Summer School in Cognitive Science
Meteyard, Bahrami, Vigliocco (2007)
Downward
bomb, drop, fall
Control
cook, sew, write
Upward
climb, rise, fly
21
Mental Simulations
Language Coextensive with Imagery
Summer School in Cognitive Science
Meteyard, Bahrami, Vigliocco (2007)
Downward
bomb, drop, fall
Control
cook, sew, write
Upward
climb, rise, fly
22
Verb Image Schemas
Language Coextensive with Imagery
Summer School in Cognitive Science

• An intimate relationship between language and
  space
• Language is embodied and metaphorical (Gibbs,
  2006 Lakoff, 1999)
• Spatial registration hypothesis (Chatterjee,
  2001 Coslett, 1999)
• Language is not describable without reference to
  cognitive processes such as attentional
  mechanisms (Langacker, 1991 Talmy, 1983)
• Perceptual symbol systems cognitive
  representations are schematics, or simulations of
  perceptual experiences (Barsalou, 1999)

Richardson, Spivey, Barsalou, McRae (2003)
23
Verb Image Schemas - Offline Data
Language Coextensive with Imagery
Summer School in Cognitive Science
Richardson, Spivey, Barsalou, McRae (2003)

• Forced Choice Norming Study

24
Verb Image Schemas
Language Coextensive with Imagery
Summer School in Cognitive Science
Richardson, Spivey, Barsalou, McRae (2003)

• Free Drawing Norming Study

25
Language Coextensive with Imagery
Summer School in Cognitive Science
26
Language Coextensive with Imagery
Summer School in Cognitive Science
27
Language Coextensive with Imagery
Summer School in Cognitive Science
28
(No Transcript)
29
Verb Image Schemas
Language Coextensive with Imagery
Summer School in Cognitive Science

• Did subjects agree with each other?
• 70 agreement in forced choice Std errors about
  10 in freeform.
• Did the tasks tap the same representation?
• Axis/aspect angle correlation, R 0.71
• Did they agree with linguistic intuitions?
• Significant effect of expected axis in both
  studies

30
Verb Image Schemas - Online Data
Language Coextensive with Imagery
Summer School in Cognitive Science
Richardson, Spivey, Barsalou, McRae (2003)

• Experiment 1 - Visual discrimination
• Hypothesis comprehending a verb will activate a
  form of spatial representation that will be
  extended along either a horizontal or vertical
  axis, as shown by norming data.
• Non-compatible mental imagery will interfere with
  the detection or discrimination of overlapping
  visual stimuli (Perky, 1910 Craver Arterberry,
  2001)
• Participants presented with a spoken sentence
  containing a horizontal or vertical verb, and
  identified a rapidly flashed visual stimulus.

31
Verb Image Schemas - Online Data
Language Coextensive with Imagery
Summer School in Cognitive Science
Richardson, Spivey, Barsalou, McRae (2003)

• Performance on the visual task was impaired when
  the visual stimulus appeared along the axis of
  the verbs image schema

32
Verb Image Schemas - Online Data
Language Coextensive with Imagery
Summer School in Cognitive Science
Richardson, Spivey, Barsalou, McRae (2003)

• Experiment 2 - Memory
• Hypothesis the spatial structure associated with
  a verb will influence the encoding of visual
  stimuli
• Imagery improves memory (Paivio, 1969), and
  visual stimuli are remembered better when they
  are presented in the same spatial locations at
  presentation and test (Santa, 1977 Zimmer 1998)
• Participants heard a sentence and saw pictures of
  the agent and patient of the sentence, presented
  centrally. At a later test stage, they were shown
  pairs of pictures and asked if they had been
  previously been presented together. At test, the
  pictures were presented in a horizontal or
  vertical alignment.

33
Verb Image Schemas - Online Data
Language Coextensive with Imagery
Summer School in Cognitive Science
Richardson, Spivey, Barsalou, McRae (2003)
34
Verb Image Schemas - Online Data
Language Coextensive with Imagery
Summer School in Cognitive Science
Richardson, Spivey, Barsalou, McRae (2003)
35
Verb Image Schemas - Online Data
Language Coextensive with Imagery
Summer School in Cognitive Science
Richardson, Spivey, Barsalou, McRae (2003)

• At test, picture pairs were recognized faster if
  they were presented in the same orientation as
  the associated verbs image schema

36
Verb Image Schemas
Language Coextensive with Imagery
Summer School in Cognitive Science
Richardson, Spivey, Barsalou, McRae (2003)

• In two offline tasks, naïve participants
  categorized verbs as having a horizontal or
  vertical image schema
• There was a good degree of consistency between
  tasks and between participants
• The offline tasks provided norms for experiments
  involving online verb comprehension
• In both a visual discrimination and a memory
  task, reaction times showed an interaction
  between the horizontal/vertical nature of the
  verbs image schema, and the horizontal/vertical
  position of the visual stimuli
• Such spatial effects of verb comprehension
  provide evidence for the perceptual-motor
  character of linguistic representations (see also
  Bergen, Lindsay, Matlock Narayanan, 2007).

37
Embodied Language
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
38
Language Coextensive with Imagery
Summer School in Cognitive Science
Iconicity Affects Semantic Relatedness Judgments
(Zwaan Yaxley, 2003)
attic basement
39
Language Coextensive with Imagery
Summer School in Cognitive Science
Iconicity Affects Semantic Relatedness Judgments
(Zwaan Yaxley, 2003)
basement attic
40
Embodied Language
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
41
Embodied Language
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
Visual Properties Influence Concepts
Motor Properties Influence Vision
Language Influences Visual Properties

Language and Motor Influence Each Other

Your Entire Body Participates in Thought

42
Embodied Language
Summer School in Cognitive Science
(Spivey, Tanenhaus, Eberhard, Sedivy, 2002)
Put the apple on the towel in the box
Ambiguous
Put the apple thats on the towel in the box
Unambiguous
One-Referent Context
Two-Referent Context
43
Embodied Language
Summer School in Cognitive Science
Motor Constraints During Language
(Chambers, Tanenhaus, Magnuson, 2004)
Pour the egg in the bowl onto the flour. Pour the
egg thats in the bowl onto the flour.
Compatible
Incompatible
44
Embodied Language
Summer School in Cognitive Science
Motor Constraints During Language
(Chambers, Tanenhaus, Magnuson, 2004)
45
Embodied Language
Summer School in Cognitive Science
Motor Facilitation During Language
The Action-Sentence Compatibility
Effect (Glenberg Kaschak, 2002)
46
Embodied Language
Summer School in Cognitive Science
Motor Facilitation During Language
The Action-Sentence Compatibility
Effect (Glenberg Kaschak, 2002)
Close the drawer. Open the drawer.
47
Embodied Language
Summer School in Cognitive Science
Motor Facilitation During Language
The Action-Sentence Compatibility
Effect (Glenberg Kaschak, 2002)
Andy delivered the pizza to you. You delivered
the pizza to Andy.
48
Embodied Language
Summer School in Cognitive Science
Motor Facilitation During Language
The Action-Sentence Compatibility
Effect (Glenberg Kaschak, 2002)
Liz told you the story. You told Liz the story.
49
Embodied Language
Summer School in Cognitive Science
Motor Facilitation During Language
The Action-Sentence Compatibility Effect
(Glenberg Kaschak, 2002)
(For variation in the temporal dynamics, see
Kaschak Borregine, 2008)
50
Embodied Language
Summer School in Cognitive Science
Motor Activation During Language
(Pulvermüller, 1999)
51
Embodied Language
Summer School in Cognitive Science
Motor Activation During Language
(Pulvermüller, 1999)
Language Networks
52
Embodied Language
Summer School in Cognitive Science
Motor Activation During Language
(Pulvermüller, 1999)
Language Networks Associated with Visual Objects
53
Embodied Language
Summer School in Cognitive Science
Motor Activation During Language
(Pulvermüller, 1999)
Language Networks Associated with Actions
54
Embodied Language
Summer School in Cognitive Science
Motor Activation During Language
(Martin, Wiggs, Ungerleider, Haxby, 1996)
55
Embodied Language
Summer School in Cognitive Science
Motor Activation During Language
(Pulvermüller, Härle, Hummel, 2001)
56
Embodied Language
Summer School in Cognitive Science
Motor Activation During Language
(Pulvermüller, Härle, Hummel, 2001)
57
Embodied Language
Summer School in Cognitive Science
Motor Activation During Language
(Pulvermüller, Härle, Hummel, 2001)
58
Embodied Cognition
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
59
Embodied Cognition
Summer School in Cognitive Science
Sensorimotor Influences in Cognition
Visual Properties Influence Concepts
Motor Properties Influence Vision
Language Influences Visual Properties

Language and Motor Influence Each Other

Your Entire Body Participates in Thought

60
Cognition Coextensive with Action
Summer School in Cognitive Science
Your Entire Body Participates in Thought
Subliminal Priming of Old Age (Bargh et al.,
1996)
Push/Pull response to pos/neg words (Chen
Bargh)
Associations with Approach or Avoid
Gestures (Cacioppo and colleagues)
Orofacial muscles influence judgments (and
IAT) (Cacioppo and colleagues)
A Fleeting Moment of Zen and movement/memory
61
Embodied Cognition
Summer School in Cognitive Science
Sensorimotor Influences in Cognition