Gesturing and Naming: Analysis of

1
Gesturing and Naming Analysis of Functional
Knowledge in Object Identification Daniel N.
Bub Michael E. J. Masson Cindy M.
Bukach University of Victoria
2
Object Identification Relation between
gestural knowledge and object identification
Neuroimaging and case studies suggest specialized
use of gestural knowledge category-specific
agnosia modeled by assuming separate systems
for structural and functional knowledge
selective activation of premotor cortex
when naming tools or imagining grasping objects
3
Martin et al. (1996). Nature.
4
Functional Knowledge Functional knowledge
appears to have privileged role in object
identification neuroimaging evidence is
correlational Seek evidence for causal
influence Define functional knowledge as
gesture used during normative interaction with
object
5
Interference Paradigm Logic of opposition
if knowledge source is necessary, it will
be applied even when negative consequences
learn color-gesture associations present
objects in congruent vs. incongruent color
6
Interference Paradigm Train specific gesture
to each of four colors
7
Experiment 1 Gesture to Color Color-object
gesture congruency tested by gestural response
to color if object automatically recruits
gestural knowledge, there should be a
congruency effect Neutral
Congruent Incongruent
(closed grasp) (poke)
(pinch)
8
Experiment 1 Gesture to Color
9
Experiment 1 Gesture to Color
Passive viewing insufficient to invoke
gestural knowledge
10
Experiment 2 Gesture to Object or Color
Require subjects to attend to object Cued to
make gestural response to object or to color
(manipulation check) Congruent
Incongruent (open
grasp) (closed grasp)
11
Experiment 2 Gesture to Object or Color
12
Experiment 3 Name Object, Gesture to Color
Functional knowledge in object naming Cued to
make gestural response to color or to name the
object Congruent
Incongruent (poke)
(closed grasp)
13
Experiment 3 Name Object, Gesture to Color
14
Experiment 3 Name Object, Gesture to Color
Distinction between functional gesture
and volumetric gesture Volumetric gesture may
be invoked during object naming For two
objects, these two types of gesture
are identical expect congruency effect
Gesture pinch
15
Experiment 3 Name Object, Gesture to Color
16
Conclusions Functional gestures invoked by
object in the context of a gesturing task set
Object naming does not appear to be
causally dependent on functional gesture
Volumetric gesture, by virtue of its relation
to structural aspects of an object, may
contribute to early identification processes
17
Next Step Disadvantage of color experiment
can only use a few objects and gestures
Develop a procedure using many objects
and gestures to investigate time course of
recruitment of knowledge about volumetric
gesture (associated with object shape, not
function) time course of recruitment of
knowledge about functional gesture (function,
not shape)
18
Next Step
Subjects trained to imitate Hand actions
19
Next Step
Functional
Volumetric
20
Next Step
Functional
Subjects must imitate Hand actions
Volumetric
21
Suppression Paradigm
Trained gesture to color red
22
Suppression Paradigm
Book Trial 2
Trial 1
Tria1 1
XXXXX
Book
Table
23
Suppression Paradigm
24
Suppression Paradigm
To produce this gesture you need to suppress
poke
25
(No Transcript)
26

Thanks very much everyone!