Mental Imagery

1
Mental Imagery

• Long-term Visual Memory

2
Outline

• When is imagery used? What is it for?
• Analog Representation (vs. Propositional)
• Sub-components of Visual Imagery
• Overlap between Imagery and Perception
• Long-term Visual Memory

3
When do we use imagery? What for?

• Which one is bigger, a tennis ball or a 100-watt
  light bulb?
• Do monkeys ears protrude from their head?
• Do robbins lay eggs? (no imagery needed)
• We use imagery to retrieve information from
  memory when the following three conditions are
  met
• The information is a subtle visual property
• The property has no verbal label (has not been
  considered before)
• The property cannot be inferred from other
  information
• We also use imagery to anticipate trajectories
• Would the billard go into the socket?
• These to functions (memory recognition, visual
  tracking) are also the primary functions of
  visual perception

4
Representations Analog or propositional? Analog
(pictorial) - maintains the spatial properties
of a visual perception. - Example Analog
representation of a square is
Propositional - Sentence-like descriptions
of the image - Example Propositional
representation of a square is (1) four
straight lines (2) at right angles to each
other (3) each line connected at the end to
two others
5
Sub-components of Visual Imagery

• Generating
• Scanning (inspection)
• Retaining
• Mental Rotation (transformation)
• Zoom (transformation)
• Evidence
• Behavioral
• Individual differences
• Brain lesion
• Spatial Imagery vs. Visual Imagery

6
Generate

• Imagine an elephant skating
• This occurs in a piecemeal fashion
• First, an image frame is generated (global
  shape)
• Next, parts and details are added to the the
  global shape, based on needs and imager
  preference
• Example Imagine a letter F

7
Mental Scanning

• Map is displayed
• Subjects memorize it
• Map is removed (imagery)
• Task Imagine black speck moving from straw hut
  to well. Press the key when you arrive
• Independent variable The distance
• Dependent variable Response Time
• Main Finding Distance scanned in a mental image
  is directly related to the time it takes to
  complete the scan

8
Image Retention

• Task Add a 1-inch line for each new command,
  recall the visual path at the end
• Verbal commands North, NE, N, W, S, W, W, S, E,
  NE
• Findings
• Difficult to retain the information
• Perceptual Chunks (e..g., W-W)
• Note that in visual perception the very rapid
  decay is a virtue (but it is a burden for
  imagery)

9
Mental Rotation
Are they the same shape?
Rotation
2-D
yes

• Stimulus A pair of 3-D objects
• Task Are the two shapes the same but viewed from
  a different perspective? Or are they different?
• Independent variables
• Rotation type (in 2-D, in depth)
• Same/different
• Angle of rotation (0 --gt 180º)
• Dependent variable Response Time
• Main Findings
• Imagined movement resembles actual movement
  (linear relation between RT and angle of
  rotation)
• This is true for both plane and depth rotations

yes
depth
no
depth
10
Zoom-in

• Instructions
• Imagine a mouse next to an elephant
• Do the mouses whiskers point up or down?
• Inspection of whiskers requires zooming-in
• Such zoom-in process is time consuming (slow RT)

11
Zoom-in (contd)

• Instructions
• Imagine a mouse next to a paper clip
• Do the mouses whiskers point up or down?
• Inspection of whiskers does NOT require
  zooming-in
• Faster RT than when paired to the elephant

12
Visual vs. Spatial Imagery

• Visuo-spatial Imagery has several sub-components
  (rotate, zoom)
• Visuo-spatial imagery can also be divided into
• Visual Imagery proper
• Unavailable to blind people
• Possibly related to introspective vividness of
  image
• Disrupted by secondary visual task (but not by
  spatial task)
• Spatial Imagery
• Evident in blind people
• Unrelated to vividness of image
• Disrupted by non-visual spatial task
• Probably more dependent on parietal cortex
• Individual differences data argue for
  independence among subcomponents (also between
  visual and spatial aspects)

13
Overlap of Imagery and Perception

• Hypothesis
• Imagery uses the same machinery as perception
• Evidence
• Behavioral (interference)
• Neuroimaging
• Brain Lesion
• But data against overlap also exists

14
Behavioral Evidence (inspection interference)

• Primary Task
• Detect a weak perceptual signal
• Secondary Task
• Imagery
• Independent variable
• Modality across tasks
• Same (high interference)
• visual-visual,
• auditory-auditory
• Different (low interference)
• visual-auditory,
• auditory-visual
• Dependent variable Perceptual Sensitivity (hit -
  FA)
• Finding There is interference
• Thought questionPredict result when perceptual
  and imaged stimuli are the same

15
Neuroimaging Evidence Similar Brain Areas
Activated by Imagery and Perception
16
Neuroimaging Evidence Topographic
correspondence between perception and Imagery
Background - Primary visual cortex has a
retinotopic map, that is, - peripheral visual
field is represented in anterior part of BA17 -
fovea is represented in posterior part of BA17
Task Letter Imagery Independent Variable
Size of image Result Image of small letters
activates more posterior parts of primary visual
cortex than image large letters.
17
Neuropsychological Evidence

• Subject Patient with occipital lesion and
  reduced visual field (vs. Normal controls)
• Instructions
• Imagine an elephant, start walking toward it
  until it covers all visual field. How far away is
  it?

Normal Visual field
Reduced Visual field
18
Neuropsychological Evidence
19
How far does the correspondence between image and
picture really go?

• Ambiguous Figure is displayed
• Subjects hold the image
• Figure is removed
• Task Reinterpret the figure
• Result Subjects are unable to reinterpret imagery

20
Imagery Conclusions

• Mental imagery can be studied in the lab
• Visual Imagery is not a monolithic concept.
• Instead, many sub-components exist (rotate,
  zoom, scan, generate)
• Mental images are a little bit like pictures
  (analogical)
• Imagery seems to rely in perceptual neural
  substrates

21

• Imagery exists in other domains too
• Motor imagery (gymnasts, skiing)

22
Visual Long-Term Memory (LTM)

• Propositional or Analog?Visual LTM is close to
• Imagery (analog)
• semantic knowledge (propositional)
• Image file
• Propositional (recipe)
• More information available than what is actively
  displayed in imagery
• Verbal labels (e.g., colors, spatial reasoning in
  Montreal vs. Seattle)
• Same rules as verbal memory
• Recency and primacy effects
• Schemas (boundary extension)
• Priming

23
Dual Coding

• Words for which visual image is easy are better
  remembered than low-imagery words (cow vs.
  virtue)
• Why?
• Possible dual coding (verbal, visual)
• verbal code is most effective for retrieving
  semantic associates
• Visual code is most effective for retrieving size
  and shape
• so, verbal and visual codes are somewhat
  different

24
Long-term Memory for Faces

• Memory is improved by rehearsal via visualization
• familiarity effects
• Same race vs. other race
• Faces from other races are more difficult to
  remember
• Synthetic face is perceived as other race
  (anglosaxons see Collin Powell black,
  african-americans see him white)
• Visual search task (dan levin)
• Eye witness ...

25
Long-term Memory for Faces Line-up

• Eye witness make a lot of errors, particularly
  when
• Suspect differs from lure across other dimensions
  (sex, attractiveness, etc.)
• Sequential presentation is better than
  simultaneous
• Metacognition and the way questions are framed
  influence performance

26
Spatial Knowledge

• Maps Navigation (B. Tversky)

27
Visuospatial Working memory

• its relation to mathematical reasoning

28
Method Reaction Time (RT)

• Manipulate stimulus -gt Measure RT
• Infer mental processes behind such S-R pattern
• Example
• Mental Rotation Are these two letters the same,
  or the mirror image of each other?

same
mirror
29
Mental Rotation What is the nature of mental
imagery? Hypothesis - Moving mental images
is like moving real objects. - For real objects,
location
Subjects were asked to judge whether the letters
were normal or mirror reversed. It was assumed
that the letters would be mentally rotated to
upright positions in order to make
this judgement.
30
Mental Rotation What is the nature of mental
imagery?

• Hypothesis
• - Moving mental images is like moving real
  objects.
• Assumptions
• Obviously, real objects move in a continuous
  fashion
• to judge whether the letters are normal or mirror
  reversed requires rotating image to upright
  positions
• Prediction

RT fast slow
31
Results
The greater the angle the letter had to be
rotated to get back to uprightthe longer it took
subjects to judge whether they were mirror
reversed or not.
32
Lessons from visual imagery

• Instrospectionism
• intuition that mental images are rotated
• Metaphoric reasoning
• Behaviorism
• test RTs
• Cognitive psychology
• Infer underlying mental processes (create image,
  rotate,inspect, zoom in, etc.)
• Elemental mental operations
• Cognitive neuroscience
• Explore underlying neural substrates (e.g., same
  areas as visual perception?)