Assessing Narrative Skills in Children

1
Assessing Narrative Skills in Children

• Peter de Villiers
• (Smith College)
• Frances Burns
• (University of Massachusetts, Amherst
• and Vanderbilt University)

Supported by NIH grant N01-DC-8-2104 web page
www.umass.edu/AAE
2
Acknowledgements

• Jill de Villiers Smith College
• Elizabeth Engen Rhode Island School for the Deaf
• Debbie Topal Rhode Island School for the Deaf
• Harry Seymour University of Massachusetts
• Barbara Pearson University of Massachusetts
• Tempe Champion University of South Florida

3
Why Assess Narrative Skills?

• Essential for continuity of personal memory,
  encoding of experiences, and social and cultural
  connection.
• A major prerequisite language skill for adequate
  reading and writing development (Snow et al,
  1998)
• A test of the productive application of syntactic
  and semantic skills in functional communicative
  contexts.
• A primary early form of extended discourse/
  taking a sustained turn decontextualized
  language with more complex syntactic forms..

4
What Aspects of Narrative to Assess?

• 1. What makes for a well-formed narrative?
• Thematic coherence on the macro-level of plot and
  episode organization.
• Linguistic cohesion or connectivity at the
  micro-level of noun phrases and clauses and their
  interrelationships across the discourse.
• Appropriate elaboration of the different points
  of view of the characters.
• 2. What specific features of these properties of
  a well-formed narrative can be easily scored and
  will translate directly into intervention?

5
Narrative Coherence

• Plot/Episode Structure the landscape of
  action (Bruner, 1986)
• Setting/Introduction Episode(s)
  Resolution/Coda
• Episode Structure
• Onset/Initiation -- introducing the problem,
  goal, or event that initiates and motivates the
  action in an episode of the story.
• Unfolding/Elaboration/Action Attempts --
  development of the action of the protagonists in
  terms of actions and attempts to solve the
  problem or reach the goal.
• Consequences -- immediate effects of each of
  these actions.
• Resolution -- the outcome of these endeavors.

6
Narrative Cohesion

• Referential Cohesion -- introducing, maintaining
  reference to, and contrasting the characters (or
  objects) in the story (Karmiloff-Smith, 1981).
• Temporal and Causal Connectivity -- clearly
  marking the time and causal relationships between
  events (Berman Slobin, 1994).
• Foregrounding and Backgrounding -- placing the
  unfolding plotline events (the foreground) in the
  context of attendant circumstances in which they
  take place (Perrera, 1986).

7
Point of View and Evaluative Commentary

• the landscape of consciousness (Bruner, 1986)
  -- talking about the mental states of the
  characters -- their emotional reactions, desires,
  and thoughts, and what they do and dont know as
  events take place.

8
Linguistic Devices in Narrative

• Referential Cohesion -- articles a and the,
  pronouns, names, adjectives, descriptive
  prepositional phrases, relative clauses.
• Temporal and Causal Connectivity -- adverbs,
  adverbial phrases, adverbial clauses.
• Foregrounding and Backgrounding -- adverbial
  clauses of time and place, often at the beginning
  of sentences.
• Point of View / Evaluative Commentary -- mental
  state words and complement clauses.

9
How to Elicit Narratives

• Open-ended stories from a topic prompt
• Familiar scripted events (e.g., a birthday
  party)
• Story retelling
• Picture or video sequences -- long or short
• How well does the elicitation technique get the
  child to produce language that incorporates the
  narrative features we have outlined AND can be
  easily evaluated and scored for those features?
• For a more complete evaluation use more than one
  type.

10
Case Study 1 Coherence and Cohesion in the
Written English Narratives of Deaf Students

• Oral subjects
• 63 eight to sixteen year olds, mean age 1110.
• Average hearing loss 95dB (range 70 to 120).
• Hearing loss onset prior to 18 months.
• Total Communication subjects
• 56 eight to sixteen year olds, mean age 123.
• 14 with deaf parents (DoD), 42 with hearing
  parents (DoH)
• Average hearing loss 99dB (range 70 to 120).
• Hearing loss onset prior to 18 months.

11
Written Narrative Samples - 1

• One multi-episode narrative based on a wordless
  childrens story -- The Pirate Story.
• This was a multi-episode story chosen because
  it had three clear episodes, each of which
  depicted an initiating event or problem, an
  action or attempt to deal with that event, and a
  resolution or consequence of the action sequence.
• The story was presented twice in the form of 16
  color slides. Then the students wrote the story
  from memory.

12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
The End
28
Pirate Story sample -- Age 96Hearing Loss
98dB, Reading Grade 1.6

• The man carrying the boat.
• The man go the water.
• I ride the boat. The man fighting to the boat.
  The man shot the gun.
• The monster chseed the man.
• The monster shot the gun.
• The man take the monyey.
• The man take bringing the boat.
• The man to boat sount.
• The man think. The man fixing the boat.

29
Pirate Story sample -- Age 125Hearing Loss
93dB, Reading Grade 34

• The men and women carrying a big ship. They
  threw ship on the water. One ship is good, other
  is bad. The ship want to go to Skull and cross
  bones ship. The ship shoot connon Skull and cross
  bones ship. The skull and crossbones ship was
  under the water. The ship the winner. The monster
  want the ship, but he didnt. The ship shoot
  arrow to moster. Monster was dead. Ship are going
  to look island. The men climb down get golds.
  They put in ship. The golds was hevey in ship.
  The ship fell in the water. Everybody swimming
  off the water. Everybody sat the island. They cut
  the tree. They fell the tree. Everybody going to
  made a new ship. The everybody o.k.
• The End.

30
Pirate Story sample -- Age 133Hearing Loss
93dB, Reading Grade 50

• There is a group of men that made a ship. They
  dicide to find a treasure. That day they were on
  the boat and travel until there was another boat.
  They had a war. The other ship lost and the
  pirate ship won. The pirate ship went to find the
  treasure. The Monster that was in the water heard
  the war and blew fire to the ship. One of the man
  killed the Monster. They went to find the
  treasure and when they got to the sandy island
  with a trap door, the men went down and took the
  treasure and left. But it was too heavy and the
  ship sank. The men swim to the island and live
  and made the ship forever.
• The End

31
Coherence -- Episode Completeness
( ) oral students
32
Written Narrative Samples - 2

• Two short narratives based on picture sequence
  scenarios.
• These were designed to motivate the need to
  identify the characters in a contrastive way, to
  express temporal and causal relationships between
  events, and to refer to their mental states in
  explaining their actions.
• They were written with the picture sequence in
  front of the students all the time.

33
The Candy Stealing Story
34
The Balloon Popping Story
35
Candy Stealing Story sample -- Age 96Hearing
Loss 98dB, Reading Grade 1.6

• He want to the candy
• The girl gave to the a penny
• The girl gave to the cookies
• The girl dont went the cookies
• The girl dreaming police
• The girl gave penny
• The woman said thank you

36
Candy Stealing Story sample -- Age 125Hearing
Loss 93dB, Reading Grade 34

• Kerian Steal Candy Bag
• Jane and Kerian went to the store. Jane like to
  buy some jelly bean in jar. Kerian saw candy in
  the shelf. Kerian want to steal some candy in the
  shelf. Then take candy and put in her purse.
  Kerian told Jane her, you want some candy, I
  steal candy bag in the shelf. Jane said no thank,
  because she learn in school. Kerian went to
  sleep. She dream about policeman take Kerian go
  to jail. Then went into the store. Kerian pay for
  Mrs. Williams. She said Im sorry I steal candy
  bag in the shelf. Mrs. Williams said, that o.k.
  you wont go to jail. Kerian feel O.K.

37
Candy Stealing Story sample -- Age 133Hearing
Loss 93dB, Reading Grade 50

• There two girls in the store and one girl with a
  pocket purse. The girl was looking at the store
  lady and was stealing some candy on the counter,
  then they left. Outside the girl ask the girl who
  was buying the candy, and she didnt wanted it.
  That night the girl had a bad dream about going
  to jail and the police took her. The next morning
  she went to the store and paid for the candy, and
  then the store lady was happy and pat her on the
  head! The girl went home happily!!

38
Cohesion-- Pronoun Use
( ) oral students
39
Reference Specification
( ) oral students
40
Temporal Links
( ) oral students
41
Partial Correlations between Reading
Comprehension level and Features of Written
Narrative (controlling for Age and Hearing Loss)
p 42
Assessing Narrative Skills in Children Case
Study 2 African-American English and Mainstream
American English Children

• Frances Burns
• University of Massachusetts, Amherst

43
Previous Narrative Research

• Only a few studies on narrative development have
  focused on children who speak a dialect other
  than mainstream American English (MAE).
• Of these studies, an even smaller number have
  focused on the discourse skills of young children
  who speak African American English (AAE)
  (Champion,1998 2003).
• Previous studies of young AAE speakers have
  focused on their overall narrative structures and
  the content of their stories (Champion 2003).

44
Narrative Style in AAE Children

• Michaels (1981), described the structure of
  African American childrens narratives as complex
  but different from those of middle class European
  American children.
• The narratives of the majority of African
  American children were seen as topic-associating
  (TA) rather than the topic-centered, linear style
  that dominates early schooling.
• Topic associating refers to a narrative style in
  which the main topic is not explicitly stated
  but implied via a number of loosely connected
  episodes.
• Topic-centered refers to a linear progression of
  information with explicit lexical temporal
  grounding and no significant shifts in
  temporal-spatial perspective.

45
Narrative Style Contrasts

• Topic-centered
• Organized around a single topic or closely
  related topics.
• Main characters and temporal/locational
  grounding remain constant and are lexically
  explicit.
• Clear thematic progression with beginning, middle
  and end.

• Topic-associating
• Organized around loosely linked topics with
  implied (associative) connections.
• Frequent shifts in key characters and
  temporal/locational grounding.
• Does not adhere to a linear pattern of
  organization.

46
Example of Topic Associating Narrative (85 -
girl)

• 1. I live on lyme street
• 2. its a nice place
• 3. I got a- my auntie lives up there
• 4. I was gonna go to my- another school
• 5. this year Im going to a different new school
• 6. so I might be happy there
• 7. but about my house
• 8. I just love being at my house
• 9. my cousins come over to play with me
• 10. an sleep over sometimes
• 11. sometimes I have slumber parties
• 12. great!
• 13. an den in the morning sometimes my mother
  takes us- my grandpa take us to the park
• 14. get us mcdonalds or ummm all of that
• 15. sometimes he take us to the zoo
• 16. an see all the animals
• 17. it was fun at the zoo
• 18. I saw the animals, bears
• 19. it was great!

47
Example of Topic Centered Narrative (610 - girl)

• 1. one day I was going over aunts house.
• 2. then me and my cousin Jenea, we wanted to go
  to the liberry.
• 3.then we got there and I was reading books.
• 4. and then I wanted to um go on computers.
• 5. so I signed up.
• 6. but then wewhich.
• 7.uh then a magic show was um startin to come on.
• 8. then this guy, he was just, he didnt know
  where his magic hat was.
• 9. so he made a hat with big balloons like
  clowns.
• 10. and then after he made a hat he made um the
  duck out of balloons.
• 11. um it was like that duck thats on Michael
  Jordan.
• 12. he made that of balloons.
• 13. an then he, he had helpers.
• 14. but he didnt pick me.
• 15. an then he, whoever go, whoever did the job
  he gave them a wand.
• 16. an then when the magic show was done we, they
  had snacks.
• 17. they had cracker fishes, cookies and juice.
• 18. then I wanted to go a computer.
• 19 but I forgot that I had to go on the computer.

48
Further Research on Topic Associating Narrative
Style

• Hyon and Sulzby (1994), looked at the narrative
  styles of 48 African American low-income urban
  kindergarteners.
• 58.3 of the narratives were topic centered.
• 33.3 were topic associating.

49
Further Research on Topic Associating Narrative
Style

• Champion (1998), found that African American
  children, ages 6-10, produced a variety of
  narrative structures including the classic
  narrative structure.
• In fact 66 of the narratives were classified as
  classic or topic-centered.
• As defined by Labov (1972), these narratives
  included an orientation, a complicating action,
  and a resolution, and then concluded with a coda.
  
• Only 11 were classified as performatives or
  topic associating.

50
Current Research, Burns (2003)Study 1
Open-Ended Narratives

• 21 typically developing African American children
  from the Northeast aged 59 to 116 (Mean age
  82).
• The participants were video-taped telling at
  least three open-ended stories to one adult
  African American listener.
• The example topics (i.e., hurt, fieldtrip, a
  hero, vacation) were provided in order to prompt
  non-fictive narratives.
• Fictitious narratives were discouraged because
  children may be tempted to tell fairytale or
  story book narratives that are limited in AAE
  features and perhaps bias the children toward
  topic-centered narratives.

51
Study 1- Data Analysis

• A total of sixty-six narratives were transcribed
  and analyzed for AAE dialect features, T-Units,
  and narrative style (topic centered vs topic
  associating).
• The children were placed on a dialect continuum
  that ranged from low to high use of AAE on the
  basis of the frequency of appearance of several
  distinctive syntactic, semantic, and phonological
  features of AAE (Washington Craig, 1998) in
  their spontaneous speech.

52
Study 1 - Results

• Only 11 of the open-ended narratives were
  categorized as topic-associating, confirming
  Champions (1998) findings.
• There was no relationship between depth of AAE
  dialect and the likelihood that the children
  would produce topic-associating forms of
  narration.
• There was a trend in the data for the younger
  children to produce more topic-associating
  narratives.
• The younger children also produced a higher
  percentage of series of events stories. In
  these there were clear topics but no orientation,
  complicating action or resolution.

53
Example of series of events (610 boy)

• 1.once upon a time I saw my friends at the beach.
• 2. it was ??? and Carmen and Carmens friend.
• 3. den we went an we ate there.
• 4. there were little pointers on the floor.
• 5. only in some ???
• 6. an I- den I we lef
• 7. den I rode my bike
• 8. den the end
• 9. oh, den I went in the house
• 10. den I got somin to drink
• 11. den I ate
• 12. den I went to bed

54
Conclusions

• These results and those of Champion (1998)
  suggest that by the time African American
  children are aged 7 or 8, they have a range of
  narrative styles available to them.
• By this age they predominantly produce the
  topic-centered, classic narrative (Labov, 1972).
• This may result from code switching into the
  style that they are exposed to in school.
• Younger African American children produce more of
  the topic-associating and series of event types
  of stories.
• More research is needed to explore whether the
  topic-associating narrative style is dominant in
  still younger children, less than age 6.

55
Current Research, Burns (2003) Study 2 Picture
Sequences

• 78 AAE (n53) and MAE (n25) speaking children.
• Ages 4 to 6. No difference between the AAE and
  MAE groups in age distribution or mean age.
• One picture sequence narrative from the
  Diagnostic Evaluation of Language
  Variation-Criterion Referenced (DELV-CR), San
  Antonio, TX The Psychological Corporation (2003)
• Assessing Reference Contrasting, Temporal Links,
  Mental State References, and Theory of Mind.
• Narratives were audio-taped and transcribed.
• SEE THE DELV-CR FOR STIMULUS PICTURES

56
Narrative Samples 1

• I want my train. Im gonna hide the train from
  him. Im gonna play out of the toy box. Im gonna
  find that train. Bring that train. (C 42, MAE)
• He was looking for the choo choo train because
  the other boy was playin. And then and then he
  said, I want that choo choo train back, and
  umm he put it in his toy box. And then he came
  back to find it and he looked under the bed and
  it wasnt there. (SC 49, MAE)

57
Narrative Examples 2

• The big boy came into the little boys room and
  took away the little boys train. Then he hid it
  under the boys bed where he couldnt get it.
  Then the little boy when he left he got out his
  train and put it in the toy box while the big boy
  was eating. Then the big boy thought about the
  train and he went under the bed to go see it but
  it wasnt there.
• (A 64, MAE)
• The little brother was trying to get his toy from
  the big brother. And the big brother hiding his
  toy under the bed. When he is eating his
  sandwich, the little boy go and get it and put it
  inside of his toy box. When his big brother walk
  in, he think about the train and he look under
  his bed for it. (J 63, AAE)

58
Study 2 - Data Analysis

• For reference contrasting, the children were
  given 1 point if they contrasted the two boys in
  some way in their story (e.g., the big brother
  vs the little boy)
• For temporal links the children were given a
  score based on the most sophisticated type of
  temporal expression they used 0 no time links
  expressed, 1 only sequencers like then or
  and then used, 2 adverbial clauses of time
  used (e.g., while or after).
• For mental state references in describing the
  thought balloon picture in the eliciting
  sequence, the children received 1 point if they
  referred to the intention or desire of the boy
  (He wants his train.), but 2 points if they
  referred to his cognitive state (He is thinking
  about his train.)

59
Study 2 - Data Analysis

• For their answers to the final question about why
  the boy was looking under the bed for the train,
  the children again received 1 point for an answer
  in terms of his motivation for looking (to find
  his train.), but 2 points for a theory of mind
  explanation (because he thinks his train is
  there.)
• So the total score on the narrative was 7 points.

60
Study 2 - Statistical Results (ANOVA)

• For reference contrasting there was a significant
  age effect (p.018), but no effect of dialect,
  and no interaction between age and dialect.
• For temporal expressions there was a significant
  age effect (p.003), but no effect of dialect,
  and no interaction between age and dialect.

61
Study 2 - Statistical Results (ANOVA)

• For mental state references in Picture 5
  descriptions, there was a significant age effect
  (p.015), but no effect of dialect, and no
  interaction between age and dialect.
• For theory of mind explanations there was a
  significant age effect (p.003), but no effect of
  dialect, and no interaction between age and
  dialect.

62
Overall Picture Sequence Narrative Scores
63
DELV-CR Field Testing Study - Typically-developin
g and Language-impaired Children

• Narratives elicited by the train story sequence.
• On-line scoring by the clinicians administering
  the test (a reliability check showed 87.5
  agreement with audio-taped and transcribed
  narratives)

64
DELV-CR Narrative Study - Subjects

• 1014 four to nine year olds from all around the
  USA.
• 60 of them speakers of AAE, 40 speakers of MAE.
• 30 of each group (roughly equally spread across
  the ages) were diagnosed as being
  language-impaired and were receiving intervention
  services.
• AAE and MAE groups were matched for parent
  education level (average level high school).

65
DELV-CR Narrative Study - Results

• No differences were found between the dialect
  groups on any of the separate measures reference
  contrasting, temporal expressions, mental state
  references, or theory of mind.
• BUT there were strong developmental growth
  effects for each of the measures.
• AND there were clear differences between the
  typically-developing children and the
  language-impaired children on each of the
  measures.

66
Development of reference contrast in narratives
(contrasting the two main characters) in
typically developing MAE and AAE speaking
children.
67
Development of reference contrast in narratives
(contrasting the two main characters) in
typically developing and language impaired
children.
68
Development of the expression of temporal links
between events in the narratives of typically
developing MAE and AAE speaking children.
69
Development of the expression of temporal links
between events in the narratives of typically
developing and language impaired children.
70
Development of mental state references to
describe the thought balloon picture (typically
developing MAE versus AAE speaking children)
71
Development of mental state references to
describe the thought balloon picture (typically
developing versus language impaired children).
72
Development of theory of mind explanations for
the characters mistaken action in the picture
narrative (typically developing MAE versus AAE
speaking children).
73
Development of theory of mind explanations for
the characters mistaken action in the picture
narrative (typically developing versus language
impaired children).
74
Development of combined narrative skills in MAE
and AAE speaking children aged 4 to 12.
75
Overall narrative scores in typically developing
and language impaired children aged 4 though 12.
76
Conclusions

• The last two studies demonstrate that these
  picture sequences produce a dialect neutral
  assessment of important features of narrative
  cohesion and point of view.
• The materials can be used for diagnosis of
  language impairment in both Mainstream American
  English speaking children and African American
  English speaking children.