What you see when you see a thing depends upon what you see the thing as But what you see the thing

1
What you see when you see a thing
depends upon what you see the thing asBut
what you see the thing as depends upon what you
know about what you are seeing
2
Here is Smith at sea on a foggy evening, and as
lost as ever he can be. Suddenly the skies clear
and Smith sees the Pole Star.
What happens next? In particular, what are t
he consequences of what Smith perceives for what
he comes to believe and do? That depends upon
what he sees the Pole star as...
If for example he sees the star that is the Cele
stial North Pole then Smith will know, to that
extent, where he is -- and we may confidently
expect the he will utter saved! and make for
port. Whereas, if he sees the Pole Star, but
takes it to be a firefly, then seeing the Pole
Star may have no particular consequences for his
behavior or his further cognitive states.
Smith will be just as lost after he sees it as h
e was before.
3
COGNITIVE ASPECTS OF VISUAL PERCEPTION
ANDPRODUCTION

• Marjorie Zielke
• October 6, 2004

4
Course Research Topics

• Visual Acquisition of Knowledge
• The Efficiency of the Visual
• Cognitive Science as one framework

5
COGNITIVE ASPECTS OF VISUAL PERCEPTION
ANDPRODUCTION

• What is Cognitive Science?
• Visual Processing vs. Cognitive Perception
• How the Brain Processes Visual Images
• Models of Thought Cognitive Perception
• Vision and the other Senses
• How the blind see and the implications..Oliver
  Sacks, New Yorker Magazine, July 28, 2003
  
• Interesting, related research at UTD School of
  Brain Sciences
  
• Future Implications
• Why have an interest in cognitive science as
  interactive developers?
  
• How can we use knowledge of the brain to design
  projects?
  
• Question and Comments

6
What is Cognitive Science?

• Multidisciplinary area which combines fields of
  neuroscience, psychology, philosophy,
  anthropology, computer science
  
• Definition The study of intelligence and
  intelligent systems, with particular reference to
  intelligent behavior as computation. (Simon
  Kaplan)
• Also referred to as the study of the mind.
• Applied Fields of study Learning and
  Conceptual Organization, Computational
  Linguistics, Psycholinguistics, Cognitive
  Neuroscience, Human Computer Interaction in
  Aviation
• Vicarious disciplineunderstand more about
  computer science/interactive development from
  studying the brain and understand more about the
  brain from studying computer science/interactive
  development.
• Personal understanding is a work in progress

7
Vision vs. Perception
8
Vision
9
Optic nerve
Optic chiasm
Optic tract
Lateral geniculate nucleus
Optic radiation
Primary visual cortex
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Perception
15
Another approach to understanding visual
perception is to develop models of how the
fluctuating patterns of light reaching the eyes
are processed to yield information about the
surrounding world, without necessarily referring
to any physiological mechanisms.
16
Ames Room
17
(No Transcript)
18
Cognitive PerceptionModels of Thought

• Bottom up, Top Down
• Behaviorists Observation
• Empiricists Perception requires the use of
  knowledge of the world to interpret the retinal
  image.
  
• Gestalt
• Marr
• Constructionist
• Connectionist

19
(No Transcript)
20
(No Transcript)
21
Gestalt

• Gestalt a number of principles of perceptual
  organization to describe how certain perceptions
  are more likely to occur than others.
  
• Proximity Things that are close together are
  grouped together
  
• Similarity Things that look similar are grouped
  together
  
• Common Fate Things that appear to move together
  are grouped together flock of birds, school of
  fish
  
• Good Continuation perceive smooth curves rather
  than irregular forms
  
• Closure one will see a closed rather than an
  open figure.

22
MarrHe was going to die

• How do we transform a pattern of light on the
  retina into awareness of the visible world.
  
• First spatial array of values of light
  intensity and hue
  
• Second A symbolic specification of the
  positions, motions and identities of surround
  objects
  
• How can the first representation be processed to
  obtain the second
  
• Marr argued that visual process is modular, a
  number of separate subprocesses.
  
• Assume that each processing stage draws on the
  representation created at the one before so that
  there is a one way flow of information.
  
• Seems at odds with empiricists tradition, which
  holds that perception requires the use of
  knowledge of the world to interpret the retinal
  image.
• But, Marr was not opposed to a role for knowledge
  or hypotheses in visual processing, and accepted
  a top-down or conceptually driven component.
  
• However, his aim was to establish the limits of
  what could be achieved through a purely bottom-up
  or data driven analysis of visual input.
  
• In this way, he argued, it would be possible to
  specify exactly the circumstances in which
  knowledge of the world is needed to resolve the
  ambiguities in visual processing, and so to avoid
  falling back on it as a general-purpose, poorly
  specified solution to problems in vision.

23
Marr used the term computational theory to
describe this aspect of his approach to visual
perception. The term does not mean a theory t
hat is just something to do w/computers.
Instead it expresses the specific and very power
ful idea that the first stage in understanding
perception is to identify the information that a
perceiver needs from the world, and the regular
properties of the world that can be incorporated
into processes for obtaining that information.
In other words, we need to know what computation
s a visual system needs to perform, before
attempting to understand how to carry them out.
24
Connectionists

• Neural Networks representations of the world are
  expressed in terms of activities that are based
  on the properties of neurons.
  
• Many modeling techniques of this kind have been
  devised, using different networks and different
  learning rules governing the changes in the
  strengths of connections between them.

25
(No Transcript)
26
How the Blind SeeThree MemoirsFrom The Minds
Eye Oliver Sacks, New Yorker Magazine, July 28,
2003

• John Hull
• Zoltan Torey
• Sabriye Tenberken

27
Synesthesia syn-es-the-sia n. Physiol.
Sensation produced at a point other thanor
remote from the point of stimulation, as of a
color from hearing acertain sound (fr. Gk, syn
together aisthesis to perceive).
Synesthesia is an involuntary joining in which
the real information of one sense is accompanied
by a perception in another sense. In addition to
being involuntary, this additional perception is
regarded by the synesthete as real, often outside
the body, instead of imagined in the mind's eye.
It also has some other interesting features that
clearly separate it from artistic fancy or purple
prose. Its reality and vividness are what make
synesthesia so interesting in its violation of
conventional perception. Synesthesia is also
fascinating because logically it should not be a
product of the human brain, where the
evolutionary trend has been for increasing
separation of function anatomically. R. Cytowic,
"Synesthesia A Union of the Senses"
Springer-Verlag, NY (p.1)
28
The world of the blindof the blindedit seems,
can be especially rich in such in-between states
the intersensory, the metamodalstates for
which we have no common language.
29
Famous Study of Creativity French Mathematician
Jacques Hadamard asked many scientists and
mathematicians, including Einstein about their
thought process
30
Einstein repliedThe physical entities which
seem to serve as elements in thought aremore or
less clear images which can be voluntarily
reproduced and combined
31
Some are of visual and some of muscular type.
Conventional words or other signs have to be
sought for laboriously only in a secondary stage
32
Nor was Einstein unique in this respectfound
that almost all scientists work this way
33
visualizing the brain as a perceptual juggling
act of interacting routines
34
Vision and the Other Senses
35
There is increasing evidence from neuroscience
for the extraordinarily rich interconnectedness
and interactions of the sensory areas of the
brain, and the difficulty, therefore, of saying
that anything is purely visual or purely
auditory, or purely anything.
36
What You See is What You Hear

• Vision is believed to dominate our multisensory
  perception of the world
  
• However, auditory information can qualitatively
  alter the perception of an unambigous visual
  stimulus to create a striking visual illusion.
  
• Findings indicate that visual perception can be
  manipulated by other sensory modalities

37
What You See is What You Hear

• When a single visual flash is accompanied by
  multiple auditory beeps, the single flash is
  incorrectly perceived as multiple flashes.
  
• Results obtained by flashing a uniform white
  disk for a variable number of times on a black
  background.
  
• Observers were asked to judge how many visual
  flashes were presented on each trial.

38
What You See is What You Hear

• Surprisingly, observers consistently and
  incorrectly reported seeing multiple flashes
  whenever a single flash was accompanied by more
  than one beep.
• Figure 1b shows that observers performance was
  the same, irrespective of whether a single flash
  was accompanied by two beeps, or two flashes by
  one or no beeps, suggesting that the illusory
  double flash is perceptually equivalent to the
  physical double flash.

39
What You See is What You Hear Nature
Magazine Dec. 14, 2000
40
Applications
41
Recognizing moving facesa psychological and
neural synthesis

• Alice J. OToole, Dana A. Roark, and Herve Abdi

42
(No Transcript)
43
(No Transcript)
44
Facial MotionWhat is it and What is it for?

• Rigid head motion can be interpreted as social
  interaction signals.
  
• To begin or end a conversation, we turn our heads
  to look at, or away from, someone.
  
• We redirect the attention of others with a
  head-turn and we nod to indicate agreement.
  
• Rigid head movements provide the observer with a
  moving stimulus and with more views of the head
  than would be encountered from a static observer
  and subject.

45
Facial MotionWhat is it and What is it for?

• Non-rigid head movements can be grouped into
  speech production movements, facial expression
  movements and eye gaze changes.
  
• The visual cues provided by the face during
  speech function to boost the intelligibility of
  speech.
  
• Facial expressions can convey a persons mood.
• Changes in the direction of eye gaze provide
  information about the object of attention.
  
• Dynamic information contributes more in poor
  viewing conditions. Because facial structure is
  a more reliable cue to recognition than the
  dynamic identity signature, motion information is
  most beneficial when viewing conditions are not
  optimal for extracting the facial structure.
• Face familiarity mediates the role of dynamic
  information in recognition. Because
  characteristic motions and gestures occur only
  intermittently, they are learned more slowly than
  static facial features.
• The relative importance of motion information to
  recognition will increase, therefore, with a
  viewers experience with a face.

46
(No Transcript)
47
Facial MotionWhat is it and What is it for?

• Subjects can discriminate individuals on the
  basis of facial motion information alone.

48
Future Implications
49
Some philosophersprofessional
artificial-intelligence critic Hubert Dreyfus for
onemaintain that achieving human-level
intelligence is impossible without a body.The
Age of Spiritual Machines,Ray Kurzweil, Penguin
Books, 1999
50
All the Senses are Important in Creating
ArtParticularly the Tactile Sense
51
Reverse Interface Concept

• Rather than try to recreate inherently human
  senses, such as touch, allow man to bring that
  ingredient to the solution and find other ways to
  integrate technology with the human interface
• A human download process
• Dreams

52
Future Applications

• Neural Implants
• Military Training
• ICT at USC

53
Comments or Questions?