The study of oral fluency development in ESL

1
The study of oral fluency development in ESL

• A colloquium on teaching and learning world
  languages
• Queens College, March 27, 2008
• Nel de Jong

2
What is fluency?

• What does it mean to say Hes fluent in French?
• What does it mean when a teacher grades a student
  on accuracy and fluency?
• Broad vs. narrow definition (Lennon, 1990)
• Broad general oral proficiency
• Narrow speed and smoothness of oral delivery
• Fluency is a characteristic of the speaker, or of
  his/her speech (product)

3
Determinants of fluency

• The fluency of the product is determined by the
  speaker
• His/her knowledge and processing
• Characteristics, like extraversion
• Its also determined by the task
• Is there time for planning?
• How complex is the task?
• How familiar is the topic?

Speaker
Task
Product
Adapted from Tavakoli Skehan (2005)
4
Model of the speaker

• What does a speaker do?
• Think of what to say
• Find the right words
• Put the words into a sentence
• Use grammar rules for word order, agreement,
  tense marking
• Pronounce the words and sentences

5
Model of the speaker
Conceptualizer
Topic familiarity and planning affect the
conceptualizer
Formulator
Grammatical encoding
Lexicon (lemmas)
A slowdown or breakdown in one of the components
will affect fluency
Phonological encoding
Topic familiarity affects the lexicon
Articulator
Levelt (1989)
6
Speech samples

• More fluent Less fluent

7
Speaker A

• I love doing sport and uh but I don't like to
  watch them in on tv so I think sports uh can
  keep your mind active and um I like some kind of
  ah sports like running and yoga and swimming and
  form for instance running uh I run I try to
  run everyday but I can't sometimes I go running
  with my roommate sometimes to the park which is
  close to our apartment and uh after that we do
  some yoga for two hours

8
Speaker B

• yes um I like sport uh because it's very good and
  very entertaining and it help a lot uh without
  exercise our body our physical uh physic so uh my
  favorite sport is soccer com uh we call it
  football but my country not over here American
  call it soccer and why I like soccer is just
  because it's the most popular uh sport in my
  country and even in the world also xxx the most
  uh popular sport in the world and I do play ad
  I play I play soccer and I like to watch it too I
  used to play back to my country when I wa when
  uh when I when I was there and uh uh

9
Speech samples

• More fluent Less fluent
• Speaker B Speaker A

10
Speaker A (2)

• everyone should have a way to do some exercises
  and exercises uh keeps your mind active for
  instance I like to do running and yoga and
  swimming I love to run every day but sometimes
  I can't so I try to run twice a week or more than
  that uh it is a good idea to have someone that
  encourages you to do something like my roommate
  my roommate always encourages me to to run or to
  to do the yoga with him

11
Speech samples

• More fluent Less fluent
• Speaker B Speaker A
• Speaker A (2)
• What happened to Speaker A?

12
Repetition

• Speaker A spoke about the same topic three times
• The third time
• Messages were already created (conceptualizer)
• Vocabulary was already activated (lexicon)
• Some grammar was activated (e.g., past tense
  forms formulator)
• The processes went more smoothly

13
Benefit of repetition
Conceptualizer
Formulator
In a repeated speech, some knowledge is already
activated, and therefore easier to access
Grammatical encoding
Lexicon (lemmas)
Phonological encoding
Articulator
14
How to measure fluencyof the speech product

• There are many different ways in which fluency
  has been measured
• Articulation rate (words/syllables per minute)
• Length, number, position of pauses
• Length of fluent runs (number of words/syllables
  between pauses)
• Phonation/time ratio ( of time filled with
  speech)
• Number of hesitations (I like to to to run)
• And more

15
How to measure fluencyof the speaker

• There are two types of knowledge
• Declarative (knowing that)
• Flexible, but slow
• Procedural (knowing how to)
• Less flexible, but fast
• When it comes to oral fluency, were interested
  in procedural knowledge
• Its fast and doesnt take up a lot of cognitive
  resources
• Declarative knowledge can be proceduralized

16
Testing proceduralization

• Proceduralization leads to higher fluency,
  because students can more easily construct longer
  and more complex sentences (Towell, Hawkins
  Bazergui, 1996). This can be measured as
• Mean Length of Fluent Run increase
• Mean Length of Pause stable or decrease
• Phonation/Time Ratio stable or increase
• Proceduralization applies to the lexicon and the
  grammatical encoder (in the formulator)

17
How to develop fluency in the classroom

• 4/3/2 Procedure
• Talk about a topic for 4 minutes
• Retell in 3 minutes
• Retell in 2 minutes
• Although students cannot repeat verbatim, they
  can benefit from recently having generated
  semantic content, and having selected vocabulary
  and syntactic constructions (Maurice, 1983
  Nation, 1989)
• Computerized version individual, no pair work

18
Research Questions

• Does repetition of a short speech increase
  fluency?
• Repetition (1 topic) vs. No Repetition (3 topics)
• If so
• What is affected?
• Proceduralization
• Speed (articulation rate)
• Is it a long-term effect?

19
Participants

• Level 4 high intermediate
• Randomly assigned
• 19 students
• 19-37 yrs (mean 25 yrs)
• L1s Arabic, Chinese, Korean, other

20
Conditions

• Two conditions
• Repetition 1 topic
• No Repetition 3 different topics

21
Procedure

• One practice session
• Three training sessions of 4/3/2 technique
• Tests Two-minute Recorded Speaking Activities
  (RSAs) about unrelated topics
• Pretest 3-4 days before training
• Immediate posttest week after training
• Delayed posttest 3.5 weeks after training

22
Preparation Take notes
23
Speaking
24
Results pre- and posttests
No Repetition (n9) No Repetition (n9) No Repetition (n9) Repetition (n10) Repetition (n10) Repetition (n10)
Pre Post1 Post2 Pre Post1 Post2
Fluent runs (syll.) 4.42 4.11 4.52 4.82
Pause length .92 1.08 1.18 .96
Phon/time ratio .59 .55 .54 .60
Syllables per minute 194 190 196 196
Proceduralization
Pause silent or filled with non-verbal fillers
(e.g., uh, um)
25
Results pre- and posttests
No Repetition (n9) No Repetition (n9) No Repetition (n9) Repetition (n10) Repetition (n10) Repetition (n10)
Pre Post1 Post2 Pre Post1 Post2
Fluent runs (syll.) 4.42 4.11 4.27 4.52 4.82 4.69
Pause length .92 1.08 .96 1.18 .96 .99
Phon/time ratio .59 .55 .57 .54 .60 .59
Syllables per minute 194 190 204 196 196 204
Proceduralization
Pause silent or filled with non-verbal fillers
(e.g., uh, um)
26
Summary of pre- and posttests results

• Some evidence for proceduralization in Repetition
  condition
• Markers of fluency
• shorter pauses
• more speech
• The effect was found on the posttest
• One week AFTER training

27
Results 4/3/2 training
Improvement on all measures
No Repetition (n9) No Repetition (n9) No Repetition (n9) Repetition (n10) Repetition (n10) Repetition (n10)
4min 3min 2min 4min 2 min 2min
Fluent runs (syll.) 4.00 4.50 4.65 4.30 4.65 5.00
Pause length .888 .835 .817 1.020 .983 .889
Phon/time ratio .56 .58 .61 .54 .57 .61
Syllables per minute 203 215 213 205 209 214
But no effect of condition
Pause silent or filled with non-verbal fillers
(e.g., uh, um)
28
Hesitations

• How about hesitations?
• Hesitations without correction
• He encourages me to to run
• There's a park by my by my apartment
• Hesitations with correction (monitoring)
• I dont like to watch sports in on TV
• When he see me when he sees me

29
Results 4/3/2 trainingHesitations
Hesitations w/o correction No effect of
repetition
No Repetition (n9) No Repetition (n9) No Repetition (n9) Repetition (n10) Repetition (n10) Repetition (n10)
4min 3min 2min 4min 2 min 2min
Hesitations without correction 5.27 6.25 5.50 5.40 5.50 5.15
Hesitations with correction 2.75 2.34 2.08 2.40 2.56 2.46
Hesitations with correction No effect of
repetition
30
Lexical variety (MSTTR)

• How about lexical variety?
• Number of different words used
• Compare
• I like pets. Cats are nice. I like cats.
• 6 types / 9 tokens .67
• I like pets, especially cats because they are
  nice
• 9 types / 9 tokens 1.00
• The Mean Segmental Type/Token Ratio (MSTTR)
  corrects for speech length
• MSTTR Mean Type/Token ratio of segments of 40
  words

31
Results 4/3/2 trainingLexical variety (MSTTR)
Type/token ratio increases for both groups
No Repetition (n9) No Repetition (n9) No Repetition (n9) Repetition (n10) Repetition (n10) Repetition (n10)
4min 3min 2min 4min 2 min 2min
MSTTR .73 .75 .76 .73 .73 .75
32
Future studies

• What linguistic knowledge is involved? (Spring
  2008)
• Vocabulary breadth and depth
• Morphosyntactic and syntactic structures
  accurate use and complexity of constructions
• Does pre-training these knowledge components
  accelerate fluency development? (Fall 2008)
• How does time pressure influence fluency
  development in the 4/3/2 task? (Spring 2008)

33
Benefits of 4/3/2 in language lab

• For students
• Improvement in fluency
• More speaking time per student
• For teachers, potentially
• Streamlining the process of collecting speech
  samples and giving feedback
• For researchers
• Streamlining data collection

34
Conclusion

• The 4/3/2 task seems to work because of
  repetition
• Repetition seems to result in proceduralization
  of knowledge (vocabulary or grammar)
• Leading to an increase in fluency
• The effect is retained for at least a month

35
Many thanks to

• Prof. C.A. Perfetti, Dr. L.K. Halderman
• Research assistants Colleen Davy, Rhonda
  McClain, Jessica Hogan
• The students and teachers at the ELI of the
  University of Pittsburgh
• Pittsburgh Science of Learning Center
• Nel de Jong, cornelia.dejong_at_qc.cuny.edu

This work was supported by the Pittsburgh Science
of Learning Center, which is funded by the
National Science Foundation award number
SBE-0354420.
36

• Extra slides

37
RSA Topics Fall 2006

• Pretest How do you feel about pets?
• Posttest Talk about a person who was very
  important to you in the past.
• Delayed posttest What is the biggest problem
  your country is facing today?

38
Open questions

• Further analyses
• Do higher-proficiency students benefit more or
  less from the 4/3/2 training than
  lower-proficiency students?
• Low intermediate vs. high intermediate
• How does 4/3/2 affect accuracy and complexity?
• Target-like use and typetoken ratio

39
Accuracy and complexity measures

• Target-like use
• Noun plurals
• Indefinite articles
• Subjectverb agreement
• Regular past tense
• Relative clauses
• Mean Segmental TypeToken Ratio (normalized for
  length of recording)

40
Open questions

• Future studies
• What is being proceduralized? What is the role of
  vocabulary and (morpho)syntactic knowledge in
  fluency?
• Can a pre-training increase the effect of the
  4/3/2 procedure?
• How do repetition and time pressure influence
  fluency development in the 4/3/2 task?

41
Repetition and Time Pressure

• Repetition enables more retrieval, thus
  decreasing working memory load, opening up
  resources to construct new and more complex
  output
• Time pressure may encourage retrieval and
  discourage construction of more complex output

Time Pressure No Time Pressure
Repetition Retrieval Retrieval and construction high cognitive resources (most headroom)
No Repetition Construction low cognitive resources (most headroom) New construction
42
Individual analyses (preliminary)

• Repetition pretest immediate posttest
• 7 out of 10 students improved pause length
  decreased with stable length of fluent run and
  phonation/time ratio
• 2 out of 10 students did not improve
• 1 student showed a trade-off between pause length
  (shorter) and length of fluent run (also shorter)

43
Individual analyses (preliminary)

• No Repetition pretest immediate posttest
• 0 out of 9 students improved
• 1 student showed a trade-off between pause length
  (longer) and length of fluent run (also longer)
• Performance of 3 out of 9 students became worse
  (shorter fluent runs, longer pauses)