Language%20and%20Perception

1
Language and Perception
Ling 411 16
2
Language and Perception 3 Topics

• The representation of phonological information
  needed for perception of syllables
• Phonological recognition vs phonological
  perception
• Influence of language on non-linguistic perception

3
Language and Perception Topics

• The representation of phonological information
  needed for perception of syllables
• Phonological recognition vs phonological
  perception
• Influence of language on non-linguistic perception

4
Demisyllables di, de, da, du
F1 and F2 For de
It is unlikely that d is represented as a unit
in perception
5
Different sizes in cortical column structure

• Minicolumn
• The smallest unit
• 70-110 neurons
• Functional column
• Variable size depends on experience
• Intermediate between minicolumn and maxicolumn
• Maxicolumn (a.k.a. column)
• 100 to a few hundred minicolumns
• Hypercolumn
• Several contiguous maxicolumns

6
Functional columns a hypothesis

• The minicolumns within a maxicolumn respond to a
  common set of features
• Functional columns are intermediate in size
  between minicolumns and maxicolumns
• Different functional columns within a maxicolumn
  are distinct because of non-shared additional
  features
• Shared within the functional column
• Not shared with the rest of the maxicolumn

Mountcastle The neurons of a maxicolumn have
certain sets of static and dynamic properties in
common, upon which others that may differ are
superimposed.
7
Similarly..

• Neurons of a hypercolumn may have similar
  response features, upon which others that differ
  may be superimposed
• Result is maxicolumns in the hypercolumn sharing
  certain basic features while differing with
  respect to others
• Such maxicolumns may be further subdivided into
  functional columns on the basis of additional
  features
• That is, this columnar structure is representing
  categories and subcategories

8
Hypercolums Modules of maxicolumns
A homotypical area in the temporal lobe of a
macaque monkey
Category (hypercolumn) Subcategory (can be
further subdivided)
9
Perceptual subcategories andcolumnar
subdivisions of larger columns

• Nodal specificity applies for maxicolumns as well
  as for minicolumns
• The adjacency hypothesis likewise applies to
  larger categories and columns
• Adjacency applies for adjacent maxicolumns
• Subcategories of a category have similar function
• Therefore their cardinal nodes should be in
  adjacent locations

10
Functional columns in phonological recognitionA
hypothesis

• Demisyllable (e.g. /de-/) activates a maxicolumn
• Different functional columns within the
  maxicolumn for syllables with this demisyllable
• /ded/, /deb/, /det/, /dek/, /den/, /del/

11
Functional columns in phonological recognitionA
hypothesis
de-
deb ded den
de- det del dek
A maxicolumn (ca. 100 minicolumns)
Divided into functional columns (Note that all
respond to /de-/)
12
Phonological hypercolumns (a hypothesis)

• Maybe we have
• Hypercolumn of contiguous maxicolumns for /e/
• With maxicolumns for /de-/, /be-/, etc.
• Each such maxicolumn subdivided into functional
  columns for different finals
• /det/, /ded/, /den/, /deb/, /dem/. /dek/
• N.B. This is a hypothesis, not proven
• But there is indirect evidence
• Maybe someday soon well be able to test with
  sensitive brain imaging

13
Adjacent maxicolumns in phonological cortex?
de-
A module of six contiguous maxicolumns
te-
be-
pe-
Hypercolum
Each of these maxicolumns is divided into
functional columns
ge-
ke-
Note that the entire module responds to -e-
14
Adjacent maxicolumns in phonological cortex?
de-
te-
deb ded den
de- det del dek
A module of six contiguous maxicolumns
be-
pe-
ge-
ke-
The entire maxicolumn responds to de-
The entire module responds to -e-
15
Functional columns in phonological recognitionA
hypothesis
REVIEW

• Demisyllable (e.g. /de-/) activates a maxicolumn
• Different functional columns within the
  maxicolumn for syllables with this demisyllable
• /ded/, /deb/, /det/, /dek/, /den/, /del/

16
Learning phonological distinctionsA hypothesis
de-
te-
deb ded den
de- det del dek
1. In learning, this hypercolumn gets
established first, responding to -e-
be-
pe-
ge-
ke-
3. The maxicolumn gets divided into functional
columns
2. It gets subdivided into maxicolumns for
demisyllables
17
Remaining problems lateral inhibition

• When a hypercolumn is first recruited, no lateral
  inhibition among its internal subdivisions
• Later, when finer distinctions are learned, they
  get reinforced by lateral inhibition
• Problem How does this work?

18
Indirect evidence for the hypothesis

• Fits the structural organization demonstrated in
  monkey vision
• Cortical structure and function have a high
  degree of uniformity
• MEG is able to pick up different locations in
  Wernickes area for different vowels
• MEG can only detect activity of at least 10,000
  contiguous apical dendrites (Papanicolaou)
• Requires perhaps at least 250 adjacent
  minicolumns
• The size of a maxicolumn or hypercolumn

19
Language and Perception Topics

• The representation of phonological information
  needed for perception of syllables
• Phonological recognition vs phonological
  perception
• Influence of language on non-linguistic perception

20
Perception Refining a simple-minded view

• Not just bottom-up
• Top-down processing fills in unsensed details
• Not confined to a single perceptual modality
• The McGurk effect
• Visual input affects auditory perception
• Conceptual structure affects auditory perception
• Not even confined to posterior cortex
• Can also use motor neurons
• Experiment left hand or right hand?
• Mirror neurons

21
Perception Refining a simple-minded view

• Not just bottom-up
• Top-down processing fills in unsensed details
• Not confined to a single perceptual modality
• The McGurk effect
• Visual input affects auditory perception
• Conceptual structure affects auditory perception
• Not even confined to posterior cortex
• Can also use motor neurons
• Experiment left hand or right hand?
• Mirror neurons

22
Top-down processing in perception
Node for CUP in conceptual area for drinking
vessels
Conceptual and perceptual information
CUP
T
MADE OF GLASS
CERAMIC
SHORT
HAS HANDLE
Visual properties are in occipital and lower
temporal areas
Properties
23
Bidirectional processing and inference
These connections are bidirectional
CUP
T
MADE OF GLASS
SHORT
CERAMIC
HANDLE
24
Bidirectional processing and inference
Thought process 1. The cardinal concept node
is activated by a subset of its property nodes
2. Feed-backward processing activates other
property nodes Consequence We apprehend
properties that are not actually present in the
sensory input
CUP
T
SHORT
HANDLE
25
Perception Refining a simple-minded view

• Perception is not just bottom-up
• Top-down processing fills in unsensed details
• It is not confined to a single perceptual
  modality
• The McGurk effect
• Visual input affects auditory perception
• Conceptual structure affects auditory perception
• It is not even confined to posterior cortex
• Can also use motor neurons
• Motor activation in speech perception
• Mirror neurons

26
The McGurk Effect
http//www.youtube.com/watch?vaFPtc8BVdJk

• Acoustic syllable ba presented to subjects
• with visual presentation of articulatory
  gestures for ga
• Subjects typically heard da or ga
• Evidence has accumulated that visual speech
  modifies activity in the auditory cortex, even in
  the primary auditory cortex.
• Mikko Sams (2006)

27
Perception depends mainly on cortical structures
already present before sensory input
Perception is hallucination constrained by
sensory data Shepherd
28
Perception Refining a simple-minded view

• Not just bottom-up
• Top-down processing fills in unsensed details
• Not confined to a single perceptual modality
• The McGurk effect
• Visual input affects auditory perception
• Conceptual structure affects auditory perception
• Not even confined to posterior cortex
• Can also use motor neurons
• Experiment left hand or right hand?
• Mirror neurons

29
Left hand or right hand?
30
Left hand or right hand?
31
Left hand or right hand?
32
Left hand or right hand?
33
Left or right hand?

• Imaging experiment
• Subjects were shown pictures of one hand
• Asked to identify left or right
• Functional imaging showed increased CBF in hand
  area of motor cortex

Peter Fox, ca. 2000
34
Motor structures in perception

• The left-hand vs. right-hand experiment
• Mirror neurons in motor cortex
• Articulation as aid to phonological perception
• Articulation in reading
• Motor activity in listening to music
• Watching an athletic event

35
Mirror Neurons

• NY Times One mystery remains What makes them
  so smart? (Jan. 10, 2006)
• Answer They are not smart in themselves
• Their apparent smartness is a result of their
  position at top of a hierarchy
• Compare
• The general of an army
• The head of a business
• Similarly, high-level conceptual nodes
• The grandmother node

36
Mirror Neurons

• What makes mirror neurons appear to be special?
• Ans. They receive input from visual perception
• The superior longitudinal fasciculus
• Connects visual perception to motor areas
• How can a motor neuron receive perceptual input?
• Motor neurons are supposed to operate top-down
• Answer bidirectional processing
• They also receive perceptual information
• Bottom-up processing

37
Superior Longitudinal Fasciculus
From O. D. Creutzfeldt, Cortex Cerebri (1995)
38
Are some neurons smarter than others?

• Claim A grandmother node would have to be very
  smart
• Identifies very complex object
• Even in many varieties
• Alternative the head of a hierarchy
• It is the hierarchy as a whole that has those
  smarts
• Similarly, mirror neurons
• They get visual input since they are connected to
  visual areas
• Superior longitudinal fasciculus

39
Implications of hierarchical organization

• Nodes at a high level in a hierarchy may give the
  appearance of being very smart
• This appearance is a consequence of their
  position at top of hierarchy
• As the top node in a hierarchy, a node has the
  processing power of the whole hierarchy
• Grandmother nodes
• Mirror neurons
• Compare
• The general of an army
• The head of a business organization

40
Multi-Modal Perception

• Perception is not just bottom-up
• Top-down processing fills in unsensed details
• It is not confined to a single perceptual
  modality
• The McGurk effect
• Visual input affects auditory perception
• Conceptual structure affects auditory perception
• It is not even confined to posterior cortex
• Can also use motor neurons
• Motor activation in speech perception
• Mirror neurons

41
A terminological problem

• We need to distinguish
• Perception narrowly conceived
• The basic process of recognition
• Single perceptual modality
• Bottom-up processing
• No motor involvement
• Perception broadly conceived
• Two different terms needed
• Recognition (a.k.a. microperception)
• Bottom-up process in a single perceptual modality
• Perception (the broad conception) (a.k.a.
  macroperception)

42
Microperception and macroperception

• Microperception
• A.k.a. recognition
• The local process of integrating features
• Performed in one perceptual modality
• Bottom-up
• Macroperception
• The overall process of perception
• Uses multiple modalities
• Uses top-down processing

43
Perceptual structures in motor production

• Perceptual structure is used in two ways
• Planning (e.g. visualizing while painting)
• Monitoring
• Examples
• Phonological recognition in speech production
• Cf. Wernickes aphasia
• Painting
• Musical production
• Baseball, soccer, tennis, etc.

44
Language and Perception Topics

• The representation of phonological information
  needed for perception of syllables
• Phonological recognition vs phonological
  perception
• Influence of language on non-linguistic perception

45
Different languages categorize differently
Examples

• Grammatical gender
• Rocks
• Plural vs. distributive
• Time as metaphorical space
• Bells
• Time

46
Grammatical Gender

• English None
• French Masculine, Feminine
• German Masculine, Feminine, Neuter
• Oneida Masculine
• Feminine-Zoic
• Feminine-Inanimate

47
Genders in Oneida (Iroquoian)

• Masculine
• Feminine-Zoic
• Women from puberty to menopause
• Animals
• Feminine-Inanimate
• Little girls
• Old women
• Inanimate objects

48
Rocks
English rock
rocks
tyhpi
tyhtyhpi Monachi
49
Bells English and French

• cloche (of a church)
• clochette (on a cow)
• sonnette (of a door)
• grelot (of a sleigh)
• timbre (on a desk)
• glas (to announce a death)

50
The Diversity of Time

• What time is it?
• How many times did you go to France?
• She spends a lot of time in front of the mirror.
• In the time of the Crusades
• At that time, she was sick.

• Quelle heure est-il?
• Combien de fois es-tu allé en France?
• Elle passe beaucoup de temps devant le miroir.
• A lepoque des Croidades
• A ce moment-là, elle était malade.

51
Language and (Visual) Perception
Phonological Phonological Object
Production Perception
Categories Vision
Language
N.B. These connections are bidirectional
52
Bidirectional connections
Review

• An established finding from neuroanatomy
• Most corticocortical connections are
• bidirectional
• Its not because the connecting nerve
• fibers (axons) are themselves bidirectional
• Its because we find different but roughly
• parallel fibers going in opposite directions

53
Recent experiments of Kay et al.

• Experiments at UC Berkeley
• Color perception do difference in color naming
  across languages influence color perception?
• Main finding
• Lateralized influence of language on perception
• Response time faster for between-category
  discrimination especially for RVF presentation
• A left hemisphere (RVF) phenomenon

green blue
54
end