Chapter 7: The Landscape of Memory: Mental Images, Maps, and Propositions

1
Chapter 7 The Landscape of Memory Mental
Images, Maps, and Propositions
2
Some Questions of Interest

• What are some of the major hypotheses regarding
  how knowledge is represented in the mind?
• What are some of the characteristics of mental
  imagery? How does knowledge representation
  benefit from both images and propositions?
• How may conceptual knowledge and expectancies
  influence the way we use images?

3

• 3 ways the brain creates meaning

4
Mental RepresentationsPictures vs. Words

• Pictures
• concrete and spatial information
• analogous to what they represent
• Words
• abstract and categorical information
• symbolic of what they represent

5
Mental Imagery

• Internal representation of items that are not
  currently being sensed
• May involve any of the sensory modalities
• Imagine a taste, a sight, a touch
• Individual differences in creating and
  manipulating mental images
• Use of mental images can help to improve memory

6
Dual-Code TheoryPaivio (1971)

• We use two codes to represent information
• Analogue (pictorial) codes
• Symbolic (verbal) codes
• Two codes are linked

7
Visual Codes Processed Differently from Symbolic
Codes

• Each type of code is affected by different
  manipulations
• Visual interferes with spatial
• Verbal interfere with spoken
• Sequence matters more for words, not so much for
  images

8
Evidence for Dual-Code Theory

• Brooks (1968)
• One group saw a block diagram of a letter
• Memorized it
• Were asked to mentally travel the letter and
  indicate if the corner was on the extreme top or
  bottom

Start
9
Evidence for Dual-Code Theory

• Brooks (1968)
• Second group saw a sentence
• Memorized it
• Were asked to classify each word as a noun by
  indicating yes or no
• Verbal task

A bird in the hand is not in the bush
10
Evidence for Dual-Code Theory

• Brooks (1968)
• Participants were then asked to respond in one of
  two ways
• Say Yes or No
• Point to the answer Yes or No
• Why was this important?

Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
Yes
No
11
Evidence for Dual-Code Theory Interference!
Task Verbal Pointing
Letter Diagrams 11.3 sec. 28.2 sec.
Sentences 13.8 sec. 9.8 sec.
For image task, RT was slower when pointing For
the symbolic task, RT was slower for the verbal
response Different pattern different processing
for different codes
12
Evidence for Dual-Code Theory

• Participants answered questions about word or
  picture pairs

Question Word Stimuli Picture Stimuli
Associated? Mouse-Cheese  
Similar size? Thimble-Acorn  
13
Longer RT to answer the size question about the
word pairs.
14
Propositional Theory

• Do not store in form of images
• Instead have a generic code that is called
  propositional
• Store the meaning of the concept
• Create a verbal or visual code by transforming
  the propositional code

15
Propositional Representations
16
Limits of mental images
17
Test Your Visual Imagery Ability!
18
(No Transcript)
19
Try Again with Another Design
20
(No Transcript)
21
Now try it with this figure
22
Ambiguous figures

• People were unable to discover a second
  interpretation from the image
• Then drew the figure and could find the other
  interpretation
• A propositional code may override the imaginal
  code

23
Labels matter, too
Hourglass or table
Sun or ships wheel
24
Results

• Participants were asked to draw items seen
• Participants distorted the images to fit the
  labels
• This pattern supports the idea that images may be
  stored propositionally, not as original analog
  image

25
Mental Imagery Studies

• Functional-Equivalency Hypothesis
• Mental images are internal representations that
  operate in a way that is analogous to the
  functioning of the perception of physical objects

26
Mental Imagery

• Shepard Metzler (1971)
• Participants had to decide whether displays had
  two similar shapes
• Some pairs were similar, but rotated to various
  degrees

27
Shepard Metzler (1971) ResultsUsing
single-cell recordings in the motor cortex, there
is physiological evidence that monkeys can do
mental rotations
28
Neuroscience and Functional Equivalence

• Activation in the frontal and parietal regions
  occurs when viewing or imagining an image
• No overlap in the areas associated with vision
• Schizophrenics have difficulty differentiating
  between internal images and perception of
  external stimuli

29
AI and Spatial Ability video

• What information is essential for navigating in
  ones environment?
• What form might this spatial information take?
• What information would a robot need to navigate
  in an environment and would this differ from what
  a human would need?
• How do animals do it?
• rats, bees, and pigeons?

30
Image Scaling

• Does the rabbit have whiskers?
• Does the rabbit have ears?
• Does the rabbit have a beak?
• Reaction time to answer is measured

31
Results

• It took longer to respond to rabbits paired with
  elephants than to rabbits paired with flies

32
Image scaling

• Asked college students and fourth-graders simple
  questions about animals
• Does a cat have claws?
• Does a cat have a head?
• Varied the type of instructions
• Imagery vs no imagery

33
Results

• In imagery condition
• questions were answered faster if the attribute
  was larger
• In no imagery condition
• questions were answered faster based on
  distinctiveness of characteristic for the animal,
  no impact of size

34
Size Judgments

• Which is larger, moose or roach?
• Which is larger, wolf or lion?
• The closer in size, the longer the reaction time

35
Map drawing activity

• West acres mall
• Red river zoo
• Island park
• City of Fargo
• City of Moorhead
• State of North Dakota
• State of Minnesota

36
Image Scanning

• Kosslyn (1983)
• Memorize map
• Later asked to scan image
• Manipulate distance between items in scan
• Hut to grasses
• Lake to hut
• Measure reaction time

37
Results

• support for functional-equivalence hypothesis
• Linear relationship between the distance to scan
  and actual reaction time of participants
• Mental images are internal representations that
  operate in a way that is analogous to the
  functioning of the perception of physical objects

38
Demand Characteristics

• Major criticisms of Kosslyns research
• Pylyshyn
• Only one code, propositional
• Results due to task demands
• Instructions imply some necessary relationship
  between the physical distance and time required

39
Demand Characteristics Mental Scanning

• Participants give the experimenters the pattern
  they expect
• Intons-Peterson replicated research but misled
  experimenters
• If experimenter expectations are part of demand
  characteristics, then leading participants to
  believe that longer distances would lead to
  faster responses should alter the results
• Evidence supported demand characteristics idea

40
Demand Characteristics Mental Scanning

• Jolicoeur Kosslyn (1985)
• Created a false demand characteristic for a
  U-shaped function for participants
• Proposed that Gestalt principle of proximity
  makes close points hard, and distant points
  would also take longer
• No experimental expectancy effect found
• Supported idea that image is being used

41
Johnson-Laird (1983)

• Proposed there are three types of mental
  representations
• Propositional representations are pieces of
  information resembling natural language
• Mental imagery are perceptual models from a
  particular point of view
• Mental models are structural analogies of the
  world

42
Characteristics of a Mental Model

• A representation of a described situation rather
  than a representation of a text itself or the
  propositions conveyed by a text
• The structure corresponds to the functional
  relations among entities as they would exist in
  the world
• A simulation of events in the world, either real
  or imaginary

43
Evidence for Mental Models

• Kerr (1983)
• Studied participants who were blind
• Created a tactile Kosslyn map study equivalent
• Participants had to study the island, given a
  physical map to touch
• Asked the same scanning questions
• Found the same pattern of resultslonger
  distances, longer reaction times

44
Visual Imagery Spatial Imagery

• Visual imagery (images are visual)
• Seeing colors
• Comparing shapes
• Spatial imagery (analog spatial format)
• Rotating objects
• Aiming and shooting at a target

45
Neuroscience Evidence

• Farah (1988)
• Brain injury case study (L.H.)
• Gave some visual tasks
• Color identification, object naming
• Gave some imagery tasks
• Mental rotation, mental scanning
• Poor visual image skill
• Normal spatial image skill
• Thus, both types of imagery must exist

46
In Sum, Researchers Have Proposed

• Evidence for analog codes
• Evidence for propositional codes
• Evidence for mental models
• Evidence for mental imagery that is spatial
• Evidence for mental imagery that is visual

47
Cognitive Maps

• Historically
• Tolman rats
• von Frisch bees
• Thorndyke humans
• Gain increased spatial knowledge
• Using three types of knowledge
• Landmark (special buildings)
• Route-road (procedures to get to one place from
  another)
• Survey (global map-like view)

48
Heuristics Affecting Cognitive Maps

• Right-angle bias
• Streets are drawn at 90-degree angles
• Symmetry heuristic
• Irregular geographic boundaries are made regular
• Rotation heuristic
• Tend to regularize tilted landmarks to
  appropriate E-W or N-S axis
• Alignment heuristic
• represent landmarks and boundaries as better
  aligned than they really are
• Relative-position heuristic
• Relative positions of landmarks and boundaries
  are distorted in ways consistent with conceptual
  knowledge