Cognitive Processes during Text Comprehension

1
Cognitive Processes during Text Comprehension

• Paul van den Broek
• University of Minnesota

2
Outline

• Components of text (discourse) comprehension
• Product
• processes
• A framework that captures the complex
  interactions between on-line processes during
  reading and the gradual construction of a memory
  representation of the text
• Connections to cognitive neuroscience

3
Computational Model
Cognitive Theory
Neuro-imaging Studies
4
1A. The product of discourse comprehension
Episodic Memory Representation

• A network representation, consisting of elements
  from text and semantic memory connected by
  semantic relations (e.g., Kintsch, 1988
  Graesser Clark, 1985 Trabasso van den Broek,
  1985)
• During reading, the reader activates information
  relevant to the current text, establishing
  semantic relations
• between parts of the text
• between text and background knowledge

5
Referential and Causal relations

• Example
• The lady gave the waiter 100
• He returned to give her the change
• In complete texts, relations are more complex
• May require background knowledge
• May extend over long distances
• May require coordination of multiple pieces of
  information
• The language may signal
• The moon exerts gravitational pull on the earth,
  and thus played an important role in the
  development of life on earth.

6
(No Transcript)
7
Memory as a function of number of connections in
network
8
Priming as a function of network distance
9
Evidence for Network Representation

• The more connections an event has, the better it
  is remembered
• Reminding someone of one part of the text primes
  parts that are related more than parts that are
  not related (e.g., naming times, lexical
  decisions, speeded recognition)
• Events with many connections are judged more
  important, more often included in summaries

10
1b. Cognitive processes during comprehension

• Limited attentional capacity
• As information becomes activated, other
  information becomes deactivated
• Fluctuating activations of concepts as reader
  proceeds through text

11
Fluctuating activations-A view from the sentence

• At each reading cycle, four possible sources of
  activation
• Current input
• Carry over from prior text
• Episodic memory representation of prior text
  (occasionally physical text)
• Semantic (background) knowledge
• Activation from these sources occurs through
  distinct processes

12
Comprehension processes

• Two mechanisms for accessing memory (Hagoort M)
• spread-of-activation (memory-based, associative)
  processes
• coherence-oriented processes
• Resulting in
• Look-backs
• Reinstatement from episodic memory
• Retrieval from semantic memory
• Connecting inferences (content in Working memory)

13
Testing hypotheses about reading processes

• Naming, speeded recognition, lexical decision
• Speed of response is indication of activation
  (e.g., OBrien Myers, 1987 McKoon Ratcliff,
  1990 Singer, 1994)
• Reading times, Gaze duration/eye fixations,
  intonation in reading aloud

14
Example Lexical Decision

• John dropped the banana peel on the floor
• Mary fell on her back
• slip
• but also eat, yellow, pain

15
Memory-based and Coherence-oriented processes

• Balancing act
• limited attentional (working memory) capacity
• Need for coherence standards of coherence
  (Hagoort C)
• Standards depend on
• Individual differences
• Reading goals/instructions
• Linguistic input properties
• genre
• syntax, vocabulary
• connectors
• Input order
• Standards of coherence pertain to both
• Which types of relations matter (causal,
  referential, others)
• The strength of the required relations
• Good enough , shallow vs. deep processing

16
2. Connecting processes and product the
Landscape Model

• The above mechanisms and processes do not take
  place in isolation
• simultaneously
• dynamically (different combinations)
• interactively (Memory-based may support/interfere
  with coherence-oriented)
• To keep track of the various processes
  computational model (cf. Hagoort)

17
1. A young knight rode through the forest. 2.
The knight was unfamiliar with the country. 3.
Suddenly, a dragon appeared. 4. The dragon was
kidnapping a beautiful princess. 5. The knight
wanted to free her. 6. He wanted to marry
her. 7. The knight hurried after the dragon. 8.
They fought for life and death, 9. Soon, the
nights armor was completely scorched. 10. At
last, the knight killed the dragon. 11. He freed
the princess. 12. The princess was very thankful
to the knight. 13. She married the knight.
18
(No Transcript)
19
Gradual emergence of a memory representation

• At each cycle, activated concepts are added (or
  strengthened) in memory representation of the
  text
• Co-activated concepts are connected (or existing
  connections strengthened) (Hagoort U)
• asymptotic (delta) rule
• as function of strength of activations of the
  concepts
• Computationally changing connection matrix
• Concepts form interconnected clusters/cohorts
• Basis for re-activation in subsequent cycles

20
Interaction process-product

• Reading is the dynamic interaction between
  process/activation and gradually emerging mental
  representation
• Over time
• Fluctuating activations
• Different processes (memory-based and
  coherence-oriented, with subprocesses)
• Changes in episodic memory representation

21
Connecting to neuroscience
Computational Model
Cognitive Theory
Neuro-imaging Studies
22
Connections to neuro-imaging studies

• By providing detailed description of component
  processes discourse models can inform
  neuro-imaging research (and, at same time, be
  tested)
• Avoid molar questions, such as where does causal
  coherence take place, where does referential
  coherence take place?
• Component processes are recruited by both, at
  different text points
• Basis for experimentation
• Insights about brain function provide constraints
  for theoretical models
• On-line record of activities can be matched to
  time course of neural activity

23
Constraints on theoretical model

• Semantic constraints influence generation of
  inferences during reading
• the stronger the constraints, the more quickly
  and reliably inference is generated
• Possible model particular inference is generated
  in particular (set of brain structures,
  constraints influence efficiency and ease
• But possible hemispheric specialization
  generation of associates under high constraints
  more LH low constraints RH (Beeman, Virtue,
  Sundermeier, Marsolek, others)

24
Neural Pathways From the Eye to the Brain
25
Divided Visual Field Paradigm
Lexical Decision Task Inference word or
non-word was presented quickly to the
participant's left or right visual field
26
Bridging Inference Example (Virtue, van den
Broek, Linderholm, 2006)

• Todd was enjoying a vacation with his friends in
  the French Riviera. He took off his shirt and
  shoes, and he went for a walk on the beach. At
  first, Todd didn't see all of the beautiful
  shells in the sand as he waded out into the
  water.
• High Constraint
• Suddenly he called out to his friends holding his
  bleeding foot.
• Low Constraint
• Suddenly he called out to his friends holding a
  seashell.
• Target Word
• cut

27
Facilitation for Low High Constraint Bridging
Inferences
28
Predictive Inference Example

• Tom and Krista were standing together holding
  hands.
• Both of them were a little nervous, but mostly
  excited about today. Tom imagined the future as
  he looked at Krista.
• High Constraint
• They were just pronounced as man and wife.
• Low Constraint
• They were just announced as college graduates.
• Target Word
• kiss

29
Facilitation for Low High Constraint Predictive
Inferences
30
Conclusions

• Inferences under different degrees of
  constraints involve different processes not same
  processes executed with different degrees of
  efficiency/ease

31
Neurological processes

• Cognitive processes in computational models can
  be matched with ongoing neuro-imaging data

32
Methods

• This is the story of a boy named Tuk who lived
  in the Artic. He wanted to show that he could be
  brave by hunting for big animals like his father
  who was a great hunter. Some people do not like
  the idea of hunting, but Tuks family relies on
  animals for most of their food and clothing
  needs
• Narrative
• 118 clauses

33
Methods

• Magnetoencephalography (MEG)
• 248 channel axial gradiometer system (Magnes
  3600WH, 4d-Neuroimaging, San Diego, CA)
• Sampling 1017.25Hz (0.1 - 400 Hz)
• Non-invasive
• Real-time recording
• Task
• Reading comprehension

34
Results Tuk the Hunter
setting
events/actions
integration (temporal,
spatial) (causal, explanations)
35
Conclusion

• During reading, multiple processes take place, in
  dynamic and interatcive ways
• Discourse models described processes during
  reading in considerable detail
• Mechanisms
• Spread-of-activation type processes
• Coherence-oriented processes
• Fluctuating activations involving
• four sources of activation
• subprocesses
• Standard of coherence modulate the mix
• E.g., Causal, referential
• Gradual emergence of coherent memory
  representation
• We have just scratched the surface of
  neurological investigations of text comprehension
• Subprocesses (memory-based activation,
  reinstatement, etc.) rather than global
  comprehension processes (causal coherence,
  spatial layout).