Images

1
Chapter 6

• Images

2
Imaging Tasks

• How many windows are there in your bedroom?
• What does a pepperoni pizza taste like?
• What does a pepperoni pizza smell like?
• What does it feel like to jump a horse?
• What are the first bars of Beethovens 9th
  symphony?
• We store images from all the senses, but most
  research is done on visual images.
• worksheet A

3
Perception and Imaging

• visual perception involves the detection,
  recognition, and discrimination of a pattern of
  light stimulation.
• visual imaging involves the encoding and
  representation of the discriminated pattern in
  the mind.

4
Background on Imaging in Philosophy and Cog Sci

• Artistotle "The soul never thinks without a
  mental image" (De Anima) - equivalent to our
  current notion of mental representation
• The ideas of 17th through 19th c. philosophy
  are direct descendants of Artistotles images.
• One extreme the various ideas imprinted on the
  sense, . . . cannot exist otherwise than in a
  mind perceiving them. George Berkeley, Of the
  Principles of Human Knowledge.

5
Background on Imaging (2)

• 20th c. analytical philosophy (Frege,
  Wittgenstein) - treated language as the
  fundamental medium of thought, and argued
  strongly against the traditional view that
  linguistic meaning derives from images in the
  mind.
• 20th c. psychology - argued against the
  subjective introspection that was involved in
  studies of imaging.

6
Background (3)

• Interest in imagery was renewed with the
• design of objective tasks that seemed to
• require the use of mental images.
• mental rotation experiments of Shepard
• mental scanning experiments of Kosslyn
• worksheet B

7
Mental Rotation

• Subjects are shown two novel visual stimuli and
  were asked whether the stimuli had the same
  shape.
• The two stimuli were rotated, either in plane or
  in depth.
• Subjects reported that they mentally rotated one
  image in their head until the two stimuli had the
  same orientation.

8
Mental Rotation (2)

• When subjects were asked to respond as quickly as
  possible, the reaction time increased with the
  angle of rotation between the shapes.
• Shepard further found that every 50 degrees of
  physical rotation required one second of mental
  rotation before subjects could respond.

9
Mental Rotation (3)

• Response
• Time

0o 45o 90o 135o 180o

• This is the result we would expect if the
  subjects
• had moved actual objects the farther the
  object has
• to be moved the longer it takes.
• So people solve the mental rotation task by
  performing
• operations on mental objects that are similar
  to those
• they would perform on actual objects.

10
Mental Scanning

• Try imaging Snoopy the dog and mentally staring
  at his feet
• Then judge the shape of his ears
• If you start at the feet, you require more time
  to respond than if you start staring at the
  center of his body
• The time to make a decision about a property at
  one location of an imaged object increases with
  each added increment in the distance scanned.
  (Kosslyn Koenig).

11
Significance of Imaging

• The rotation and scanning experiments supported
  the claim that our experienced images are spatial
  entities and their spatial properties have real
  consequences for some forms of information
  processing.

12
Vision (Perception)
13
Vision (2)

• For a clear visual image to form
• light rays that enter the eye must come to a
  point on the retina
• the cornea and the lens bend (refract) the light
  so that the rays converge on the point of
  sharpest vision in the retina

14
Physical facts and visual perception

• The eyes lens gives the image to the retina
  upside down, reversed left to right, and flat

Retina
Lens
15
Interpreting the visual facts

• But the brain interprets retinal images as they
  really are
• The ability to interpret retinal images
  right-side up, unreversed, and in depth comes
  from experience that begins at birth
• ? Learning affects perception
• wksht B,C

16
The Brain is active in perception

• We have a general tendency to perceive objects as
  complete and unified. We fill in the missing
  parts.
• We have a strong tendency to perceive objects as
  constant in size, shape, color. (We dont see
  gray oranges in dim light.)
• Some objects may be perceived as inconstant.
  (Cubes on worksheet (D))

17
Depth Perception
Occipital Lobe
18
Depth Perception

• Because the eyes are set slightly apart, each one
  sees objects from a slightly different angle.
• So each eye sends a slightly different message to
  the occipital lobe of the brain.
• Some of the nerve fibers from each eye cross over
  at the optic chiasm
• So each side of the brain receives messages from
  both eyes.

19
Cognition

• The result of (learned) perception is a visual
  image
• Mental rotation and scanning demonstrate that we
  can store these images, retrieve them, and
  manipulate them

20
Manipulation/Computation of Images

• Visual images are useful for tasks that depend on
  visual appearance or spatial relations.
• Inspecting Imagine a plate with a knife on its
  left and a fork on its right. Is the knife to
  the left or right of the fork?
• Finding Where is your copy of Thagard?
• Zooming Image a tiny honeybee. Now, what size
  is its head?

21
Manipulation/Computation of Images (2)

• Visual imaging tasks (cont.)
• Rotating What does a capital E look like when
  it is flat on its back?
• Transforming
• Imagine the letter B.
• Rotate it 90o counterclockwise
• Put a triangle the same width as the rotated B
  directly below it and pointing down.
• What do you see? Finke, Pinker, Farah

22
Manipulation/Computation of Images (3)

• Not all tasks are amenable to imaging
• abstracting A desire for fairness
• understanding general information
• Dinosaurs are extinct
• processing causal relations
• Smoking causes cancer
• applying general rules
• Does your right shoe have the same number
  of holes as your left shoe?

23
Imaging in higher level processing

• planning
• giving following directions - a mental map
• design and construction
• explanation
• teaching of geometry
• account of continental drift
• learning
• practicing by mental imaging improves athletic
  ability

24
Computational Theory of Visual Imagery

• Kosslyn (1980,1992) develops a theory of visual
  imagery based on an analogy with computer
  graphics.
• Computer graphics files store information in a
  compressed, non-pictorial form, but when they are
  displayed they are translated into a mathematical
  map (bitmap) of the computer monitor screen, that
  specifies the color at each pixel (tiny dot) on
  the screen itself.

25
Computational Theoryof Visual Imagery (2)

• Kosslyn suggests that visual information may be
  stored in the brain as compact descriptions
• we experience an image only when this information
  is used to create a two dimensional map of visual
  space in a special, functionally defined memory
  area he calls the "visual buffer".
• What we experience as imagery, and what is
  available to the cognitive processes that use
  imagery, is the functional picture, the
  mathematical map, in the visual buffer.

26
Computational Theoryof Visual Imagery (3)

• There is a fundamental difference between
  Kosslyns computer implementation and human image
  processing
• the computer implementation does serial
  processing
• the human does parallel processing

27
Neurological Evidence Cognitive Performance
(Imaging) parallels Perception

• Evidence from deficits
• damage to the occipital lobe impairs visual
  imaging
• patients unable to see one side of space are
  unable to imagine that same side
• a patient who could not see shapes could not
  decide if George Washington had a beard
• a patient who could not process spatial relations
  could not decide how to get from one location to
  another
• Kosslyn Koenig

28
Neurological Evidence Cognitive Performance
(Imaging) parallels Perception

• Evidence from brain activity measurement
• in a technique called rCBF (regional cerebral
  blood flow), subjects asked to do tasks typical
  of imaging (finding, rotating) showed blood flow
  in the occipital lobe and other areas of the
  brain used in visual perception whereas they did
  not show this blood flow when asked to do
  mathematical or verbal tasks.

29
Underlying Questions that Remain

• How are we able to recognize resemblance between,
  e.g., a photograph and the object it represents?
  
• How is it possible for us to have the power of
  being able to mentally represent things
• (i.e., how do we get from the occipital lobe
  to the mind)?

30

• Finke, Pinker, and Farah. 1989. Reinterpreting
  visual patterns in mental imagery.
• Cognitive Science 13, 51-78.
• Kosslyn, S. 1980. Image and Mind. Cambridge, MA
  Harvard University Press.
• Kosslyn, S. O. Koenig. 1992. Wet Mind. The
  New Cognitive Neuroscience. The Free Press.
• Shepard, R. N. Cooper, L. A. 1982. Mental
  images and their transformations. Cambridge MIT
  Press/Bradford.
• Thomas, N. 2001. Philosophical issues about
  mental imagery. Macmillan/Nature Encyclopedia of
  Cognitive Science.

31
Which is better?

• I put a / after the words that were words.
  Otherwise I put a z after non-words.
• The experiment was designed to test whether
  subjects recognize words more quickly if they
  have been primed with a lexically associated
  word. I was shown a single string of letters
  followed by a second string. I was asked to
  respond one way if the second string was a word
  and a different way if it was not.