Neurological disorders of embodied and multimodal communication

1
Neurological disorders of embodied and multimodal
communication

• Elisabeth Ahlsén
• Department of Linguistics SSKKII Center for
  Cognitive Science, Göteborg University
• ZiF Center for Interdisciplinary Research,
  Bielefeld University

2
Contents

• What types of models and frameworks are used in
  work with communication disorders today and what
  are the main assumptions behind them?
• What are the present views of embodied
  communication like? - some frameworks and trends,
  models and findings
• What are the consequences of applying these
  views of communication to communication
  disorders? What can be questioned,
  changed/revised, replaced, removed or introduced?

3
Frameworks for working with communication
disorders today?

• Some examples
• Classical serial production and perception models
  (still very popular)
• Classical structuralist systems of categories
• Frameworks for aphasia classification (Boston,
  Luria)
• To some extent also
• Cognitive linguistics (to some extent)
• Conversation Analysis (to some extent)
• Pragmatics Speech act theory etc (to some
  extent)

4
Assumptions, except for pragmatic/social part

• Serial production and perception processes in
  humans (e.g. Levelt 1989 model)
• (although many features of comm.disorders point
  to more integrated processing models as more
  adequate)
• Symbol manipulation ideas, units such as
  inventories of phonemes and morphemes important
• More or less simplified localization models -
  (made better and worse by neuroimaging studies)

5
Embodied communication - trends and ideas

• - Alignment in communication (Pickering
  Garrod)
• Mirror neurons (Rizzolatti et al) (Arbib)
  (Gallese Lakoff) - extensions of this
• Coupling (Barresi)
• Resonance, Entrainment, Contagion
• - Importance of imitation and pantomime
• - Automatic processing
• Evolutionary models revisited (Deacon)
• Levels or degrees of conscious control

6
Embodied communication - different aspects

• The role of embodiment in communication and its
  importance for
• phylogenetic, as well as
• ontogenetic development, possibly also for
• microgenesis (i.e. the unfolding of a
  communicative contribution) and
• macrogenesis (i.e. conventionalization of
  communication in society) is attracting an
  increased interest.
• This interest is to some extent caused by
  hypotheses and findings concerning mirror neurons
  (cf Arbib, 2005. Gallese Lakoff 2005).

7
Arbib

• Brocas area developed atop mirror neuron system
  for grasping
• Role of imitation (simple, complex)
• Language - change from action-object frames to
  verb-argument structures
• (Cf. McNeilage Frame-Content - speech directly)
• Link to cognitive grammar (construction grammar)
• Close relation aphasia-apraxia

8
Gallese Lakoff

• Concepts are the elementary units of reason and
  linguistic meaning. They are conventional and
  relatively stable. As such, they must somehow be
  the result of neural activity in the brain. The
  questions areWhere? and How?
• A common philosophical position is that all
  conceptseven concepts about action and
  perceptionare symbolic and abstract, and
  therefore must be implemented outsidethe brains
  sensory-motor system.
• We will propose that the sensory-motor system
  has the right kind of structure to characterise
  both sensory-motor and more abstract concepts.
  Central to this picture are the neural theory of
  language and the theory of cogs, according to
  which,
• Brain structures in the sensory-motor regions
  are exploited to characterise the so-called
  abstractconcepts that constitute the meanings
  of grammatical constructions and general
  inference patterns.

9
Pickering and Garrod

• Alignment - Routines - Imitation
• Includes alignment of same person as speaker and
  listener
• Priming basic
• Speech and gestures
• Relation speech/language - praxis
• What is more automatized - more controlled

10
Feedback subproject - ZiF

• What is face-to-face communication like?

11
Data analysis (ongoing)
30 interacting pairs of students, systematically
varied with respect to sex and mutual
acquaintance Task to find out as much as
possible about each other within 3 min
Self-reported rapport

• L (000013) Bist du im ersten Semester
• R (000015) Ich bin eigentlich im fünften
  Semester aber die ersten zwei hab ich nicht
  wirklich was gemacht und dann //
• L (000018) aha
• R (000020) jetzt bin ich im dritten
• L (000023) Zoologie oder Botaniker oder was
• R (000026) entweder Anthro oder Zoologie das
  weiss ich noch nicht so genau
• L (000028) aha die Anthropologen sind viel
  besser
• L (000033) mhm /// hast du schon den Seidler
  gemacht
• R (000035) ja
• .

Max
Feedback Max
Outlook
Feedback
12
(No Transcript)
13
Communication disorders

• Time for reinterpretation
• Perception vs action not strictly
  posterior-anterior - more complex or in some
  respects more simple system
• Concrete (iconic, indexical) vs abstract
  (symbolic) - more focus on relations?

14
Examples of potential reinterpretations

• Example 1) Area F4, F5 - Brocas area apraxia
  and aphasia - link?
• Example 2) Area F4, F5 - apraxia and lack of ToM
  link?
• Example 3) Brocas area anomia - concept forming
  disorder?

15
Automatic and controlled processing in
communication

• - Mirroring - imitation - coactivation -
  alignment as central
• - Interaction basic - same things activated in
  both speakers (close link motor-perc systems)
• - Role of context, experience etc crucial
• - The whole picture - concrete vs abstract in
  semantics (Gallese Lakoff), grammar (Arbib)

16
The example of Brocas aphasia and apraxia
17
Apraxia

• Inability to perform voluntary/intended
  movements, with (and without) tools, imitation?
• either loss of idea of movement-inferior parietal
  area?, SMA, insula (SPGI- superior tip of the
  precentral gyrus of the insula)? - or of
  performance (motor programs) - premotor area
• Ideational, Ideomotor/limb apraxia, Oral apraxia
• Verbal apraxia/speech apraxia

18
Apraxia of speech

• Darley Apraxia of speech - 100 years of
  terminological confusion
• Relation to Brocas aphasia?
• Often cooccur - close localization?
• Part of Brocas aphasia?

19
Brocas aphasia and apraxia- are we still
confused?

• Still uncertainty about areas involved and their
  roles
• Still uncertainty of basic function and basic
  disturbance

20
Other relevant theories

• Motor theory of speech perception (Lieberman)
• Automatic vs propositional speech and action
  (Jackson) - in more recent versions
• - difference in apraxia and Brocas aphasia

21
Questions

• So what can ideas and findings about embodiment
  add to better understanding?
• How should we integrate an analysis of
  gestures with reasonable conceptions of apraxias
  and Brocas aphasia, other types of aphasia?
• What is the role of movements/actions? What is
  the role of verbs? Relation?

22
Brocas area - possible functions

• Brocas aphasia and apraxia often cooccur -
  normal case? Dissociation
• Basic disturbance of action-object frame -gt also
  verb-argument frame? Manual, oral and speech
  gestures? Imitation disturbed.
• Quite automatized processing - production,
  perception through simulation?
• Propositional language - sentences - Verbs?
• Brocas area in complex semantic and syntactic
  processing - LTM access?

23
Combined frameworks - Deacon

• Associationism and Holism less of two
  alternatives than two complementary aspects of a
  single process
• Both only give description of movement or
  change of information in cortical systems, since
  they fail to recognize this
• Reformulation - centrifugal and centripetal
  processes, cortically and cortex-subcortex - more
  general comprehensive model of brain function
• Based on recent neuroanatomical findings
• Basic assumptions - higher-lower functions,
  forward-backward direction, input-output will all
  need to be reexamined.

24
Combined frameworks

• Connectionism
• Anterior cortex backward connections
• Posterior cortex forward connections
• Microgenesis
• Anterior and Posterior systems in parallel from
  limbic to primary areas
• Deacon different cell layers, neurons project
  differently - centrifugal and centripetal laminar
  patterns
• .

25

• Tiers - cortex
• - Peripherally specialized areas (P)
• - Belt areas (B)
• - Association areas (A) centripetal centrifuga
  l
• - Limbic areas (L)
• Centripetal Principal thalamic inputs to layers
  iii and iv from the peripheral systems
• Centrifugal Cortical output from layers v and vi
  to subcortical sites
• Centrifugal Limbic or intralaminar thalamic
  inputs to layers i or vi

26
An integrated perspective?