Exploring Linguistic Profiles of Heritage Speakers of Spanish and Russian

1
Exploring Linguistic Profiles of Heritage
Speakers of Spanish and Russian

• A NHLRC/ACTFL Project
• Sixth Heritage Language Research Institute
• Dr. Cynthia Martin

2
Agenda

• Background
• The ACTFL OPI Assessment Criteria
• Data collection
• Preliminary findings
• Implications for instruction and assessment
• Future research

3
National Heritage Language Resource Center

• Who is the Heritage Language Learner?
• Narrow definition those who have been exposed to
  a particular language in childhood but did not
  learn it to full capacity because another
  language became dominant. FUNCTIONAL PROFICIENCY
• Broad definition those who have been raised with
  a strong cultural connection to a particular
  language, usually through family interaction.
  CULTURAL AFINITY
• Source Polinsky and Kagan 2007

4
National Heritage Language Resource Center

• HLL A generic profile
• Second or 1.5 generation
• Sequential bilingual (only/mostly HL till age 5)
• Continued use of HL (in limited ways)
• HL specific motivations and identities

5
National Heritage Language Resource Center

• Limited language exposure results in
• Incomplete grammar
• Limited vocabulary
• Limited pragmatic competence
• BUT near-native pronunciation and fluency are in
  a high range

6
National Heritage Language Resource Center

• The NHLRC/ACTFL OPI Project
• The goal is to gain an understanding of the
  linguistic, exposure, and experiential factors
  that contribute to HLLs speaking proficiency.
• Languages Russian and Spanish

7
National Heritage Language Resource Center

• Expected Outcomes
• Describe the range of oral proficiency profiles
  of heritage speakers by level and linguistic
  biographies.
• Annotate descriptors within the ACTFL Proficiency
  Guidelines Speaking to encompass the range of
  profiles that heritage speakers demonstrate.
• Enhance OPI tester training to assure that
  testers are accurately assessing the unique
  profiles of some heritage speakers
• Inform instructional practices that target the
  linguistic strengths and weaknesses of heritage
  speakers

8
ACTFL Personnel

• Project Director Elvira Swender, Director,
  ACTFL Professional Programs
• Project Evaluator Ray T. Clifford, Brigham Young
  University
• Russian Language Specialist Cindy Martin,
  University of Maryland
• Spanish Language Specialist Mildred Martinez
  Rivera, ACTFL Consultant
• Project Coordinator Jeanmarie O'Leary, ACTFL

9
Preparation for the Study

• Identify populations of heritage speakers
• Establish heritage criteria
• Screening tool
• Spanish
• 593 responses
• 162 qualified
• Russian
• 575 responses
• 132 qualified
• Demographic survey

10
Heritage Criteria

• Learned heritage language in an informal setting
• home, community, etc.
• Uses heritage language with family, friends,
  co-workers, etc.
• Lives in US
• Has received majority of formal education in
  English-speaking institutions
• 18 years or older
• Intermediate, Advanced, or Superior speakers

11
Challenges

• Identifying heritage speakers who met all the
  criteria
• Were willing to complete the background survey
• Were willing to take an OPIc
• Especially at the Intermediate level
• Sample sizes
• Spanish (41)
• Russian (50)

12
What is the ACTFL Rating Scale?

• Hierarchy of global tasks
• Four major levels
• Major levels divided into sublevels

ACTFL Rating Scale
13
What are the major levels? How are they
defined?

• SUPERIOR

• ADVANCED

• INTERMEDIATE

NOVICE
14
What are the major levels? How are they
defined?

• SUPERIOR

• ADVANCED

• INTERMEDIATE

NOVICE
Can communicate minimally with formulaic and rote utterance, lists and phrases
15
What are the major levels? How are they
defined?

• SUPERIOR

• ADVANCED

• INTERMEDIATE

Can create with language, ask and answer simple questions on familiar topics, and handle a simple situation or transaction
NOVICE
Can communicate minimally with formulaic and rote utterance, lists and phrases
16
What are the major levels? How are they
defined?

• SUPERIOR

Can narrate and describe in all major time frames and handle a situation with a complication

• ADVANCED

• INTERMEDIATE

Can create with language, ask and answer simple questions on familiar topics, and handle a simple situation or transaction
NOVICE
Can communicate minimally with formulaic and rote utterance, lists and phrases
17
What are the major levels? How are
they defined?
Can support opinion, hypothesize, discuss topics concretely and abstractly, and handle a linguistically unfamiliar situation.

• SUPERIOR

Can narrate and describe in all major time frames and handle a situation with a complication

• ADVANCED

• INTERMEDIATE

Can create with language, ask and answer simple questions on familiar topics, and handle a simple situation or transaction
NOVICE
Can communicate minimally with formulaic and rote utterance, lists and phrases
18
Assessment Criteria-Speaking
Proficiency Level
Superior
Advanced
Intermediate
Novice
Global Tasks and Functions
Discuss topics extensively, supports opinions and hypothesize. Deal with a linguistically unfamiliar situation.
Narrate and describe in major time frames and deal effectively with unanticipated complication.
Create with language, initiate, maintain, and bring to a close simple conversations by asking and responding to simple questions.
Communicate minimally with formulaic and rote utterances, list and phrases.
Context/ Content
Most formal and informal settings/ Wide range of general interest topics and some special fields of interest and expertise
Most informal and some formal settings/ Topics of personal and general interest
Some informal settings and limited number of transactional situations/ Predictable, familiar topics related to daily activities.
Most common informal settings/ Most common aspects of daily life.
Accuracy/ Comprehensibility
No pattern of errors in basic structures. Errors virtually never interfere with communication or distract the native speaker from the message
Understood without difficulty by speakers unaccustomed to dealing with non-native speakers
Understood, with some repetition, by speakers accustomed to dealing with non-native speakers.
May be difficult to understand, even for speaker accustomed to dealing with non-native speakers
Text Type
Extended discourse
Paragraphs
Discrete sentences
Individual words and phrases
19
The contexts expand as we move up the scale.

• Superior

the world
community
daily life
self
20
How do we define
accuracy/comprehensibility?

• By the level of precision needed to convey the
  message successfully
• i.e. On a continuum of structural control
• By the type of interlocutor who is able to
  understand the speaker
• i.e. On a continuum of required listener
  empathy

21
Text Type

• Text oral discourse organization
• What kind of text is required to perform the
  function?
• Words and phrases
• Simple sentences
• Oral paragraphs
• Extended discourse

22
ACTFL Proficiency Level Intermediate

• Speakers at the Intermediate level are
  distinguished primarily by their ability to
  create with the language when talking about
  familiar topics related to their daily life.
  They are able to recombine learned material in
  order to express personal meaning.
  Intermediate-level speakers can ask simple
  questions and can handle a straightforward
  survival situation.
• They produce sentence-level language, ranging
  from discrete sentences to strings of sentences,
  typically in present time. Intermediate-level
  speakers are understood by interlocutors who are
  accustomed to dealing with non-native learners of
  the language.

23
ACTFL Proficiency Level Advanced

• Speakers at the Advanced level engage in
  conversation in a clearly participatory manner in
  order to communicate information on
  autobiographical topics, as well as topics of
  community, national, or international interest.
  The topics are handled concretely by means of
  narration and description in the major times
  frames of past, present, and future. These
  speakers can also deal with a social situation
  with an unexpected complication. The language of
  Advanced-level speakers is abundant, the oral
  paragraph being the measure of Advanced-level
  length and discourse.
• Advanced-level speakers have sufficient control
  of basic structures and generic vocabulary to be
  understood by native speakers of the language,
  including those unaccustomed to non-native
  speech.

24
ACTFL Proficiency Level Superior

• Speakers at the Superior level are able to
  communicate with accuracy and fluency in order to
  participate fully and effectively in
  conversations on a variety of topics in formal
  and informal settings from both concrete and
  abstract perspectives. They discuss their
  interests and special fields of competence,
  explain complex matters in detail, and provide
  lengthy and coherent narrations, all with ease,
  fluency, and accuracy. They present their
  opinions on a number of issues of interest to
  them, such as social and political issues, and
  provide structured argument to support these
  opinions. They are able to construct and develop
  hypotheses to explore alternative possibilities.
  
•  
• When appropriate, these speakers use extended
  discourse without unnaturally lengthy hesitation
  to make their point, even when engaged in
  abstract elaborations. Such discourse, while
  coherent, may still be influenced by language
  patterns other than those of the target language.
  Superior-level speakers employ a variety of
  interactive and discourse strategies, such as
  turn-taking and separating main ideas from
  supporting information through the use of
  syntactic, lexical, and phonetic devices.
•  
• Speakers at the Superior level demonstrate no
  pattern of error in the use of basic structures,
  although they may make sporadic errors,
  particularly in low-frequency structures and in
  complex high-frequency structures. Such errors,
  if they do occur, do not distract the native
  interlocutor or interfere with communication.

25
Sublevels
26

The LOW sublevels a baseline performance for
the level

• sustained but skeletal for the level
• just hanging on

27
The MID sublevels solid performance for the
level

• quantity and quality for the level
• may have some features of the next level

28
The HIGH sublevels performance most of the time
at the next major level

• functions much of the time at the next higher
  level
• fall from the next higher level above

29
Evaluation Tools

• ACTFL OPIc
• Internet-delivered version of the ACTFL Oral
  Proficiency Interview (OPI)
• Fixed form
• All participants received the same prompts
• Intermediate through Superior-level tasks
• Same topics and role play situations
• Blindly double rated by certified ACTFL OPIc
  raters
• Samples are digitally recorded and archived
• Participants received an official ACTFL OPIc
  certificate and 25.00

30
Spanish Rater Site
31
Russian Rater Site
32
Evaluating the Sample

• OPIc sample rated holistically
• Performance of each task across the assessment
  criteria for the level
• Functions/global tasks, text type, accuracy
• ACTFL OPI rating assigned

33
Evaluating the Sample

• OPIc sample evaluated in terms of performance at
  the next higher level
• Sample evaluated in terms of specific factors
  preventing speaker from being rated at the next
  higher level
• Functional breakdown
• Specific linguistic features that were inadequate
  for the criteria of the next higher level
• Fluency (rate of delivery expected for the
  level), pronunciation, vocabulary, accuracy
  (grammar and structure) pragmatic competence,
  sociolinguistic competence, text organization

34
Intermediate Rater Review Form
35
Advanced Rater Review Form
36
Superior Rater Review Form
37
Spanish Data
38
Spanish Data
39
Spanish Data
40
Spanish Data
41
Spanish findings self-assessment

• General tendency was to over assess proficiency
  level
• Half of the Intermediate speakers self-assessed
  at Advanced
• Half (52) of the Advanced speakers self-assessed
  as Superior
• Most Superior-level self-assessed correctly (78)
• Only 11 self-assessed lower than Superior

42
Spanish findings Intermediate level

• When attempting tasks at the Advanced level
• Most could initiate but not complete the tasks
• Text type lacking in connectors and organization
• Lack of control over major time
• Limited in ability to speak about topics beyond
  the autobiographical

43
Spanish findings Intermediate level

• When attempting tasks at the Advanced level
• Least successful was talking about current event
• Requires narration and description and the
  vocabulary to move beyond the personal
• More successful was dealing with a situation with
  a complication
• Most successful was past narration

44
Spanish findings Advanced Low and Mid

• When attempting to discuss a topic from an
  abstract perspective at the Superior level
• 86 do not deal with topic abstractly
• 55 initiate task but cannot complete
• 41 revert to examples of personal experience
• 0 able to produce well organized extended
  discourse

45
Spanish findings Advanced Low and Mid

• When attempting a supported opinion at the
  Superior level
• 50 do not address the task
• 27 resort to personal experience
• 64 lack precise vocabulary
• 91 unable to produce well organized discourse

46
Spanish findings Advanced Low and Mid

• When attempting to hypothesize at the Superior
  level
• 0 were able to address the task
• 27 revert to describing a personal experience
• 86 failed to produce well organized extended
  discourse

47
Spanish findings Advanced High

• Primary reason for AH rating is functional,
  rather than a structural breakdown
• Limited ability to develop abstract ideas while
  elaborating internally cohesive messages
• When present, patterns of error are similar to
  those made by L2 learners
• Control over formulations that allow speakers to
  speculate and elaborate on outcomes and
  consequences (i.e. uses of subjunctive and other
  complex grammatical structures)

48
Spanish findings Advanced High

• When attempting to discuss a topic from an
  abstract perspective at the Superior level
• 86 initiate response but cannot complete
• 57 revert to examples of personal experience
• 72 lack extended discourse
• 14 lack precise vocabulary

49
Spanish findings Advanced High

• When attempting a supported opinion at the
  Superior level
• 57 initiate but cannot complete the task
• 29 resort to personal experience
• 57 lack precise vocabulary
• 86 unable to produce well organized discourse

50
Spanish findings Advanced High

• When attempting to hypothesize at the Superior
  level
• 0 were able to address the task
• 38 initiate but cannot complete
• 55 revert to describing a personal experience
• 55 are unable to produce well organized discourse

51
Russian Data
52
Russian Data
53
Russian Data
54
Russian Data
55
Russian findings Self-assessment

• General tendency was to over-assess proficiency
  level
• All of the Intermediate speakers self assessed at
  Advanced
• 22 of the Advanced speakers self-assessed as
  Superior
• 43 of Superior level self-assessed correctly
• 43 of Superior self-assessed lower at Advanced
• 14 of Superior self-assessed as Distinguished

56
Russian findings Intermediate level

• When asked to deal with Advanced-level tasks
• Most do not address the task
• Some initiate but are not able to complete
• None maintain oral paragraph discourse
• Most responses are marked by English interference
• Half are marked by lack of structural control and
  lack of appropriate vocabulary

57
Russian findings Intermediate level

• When attempting tasks at Advanced level
• Talking about a current event is least successful
• Cohesive and organized text is the least
  controlled feature
• Most successful task is past narration
• Russian past tense verbs are rather simple

58
Russian findings Advanced Low and Mid

• When attempting to discuss a topic from an
  abstract perspective at the Superior level
• 50 cannot deal with topic abstractly
• 88 initiate task but cannot complete
• None demonstrated communicative strategies to
  address task at Superior
• 83 revert to examples of personal experience
• 38 lack highly precise vocabulary
• 0 able to produce well-organized extended
  discourse
• Most comfortable speaking in oral paragraphs
• Vocabulary deficiencies alone do not keep these
  speakers from the Superior level

59
Russian findings Advanced Low and Mid

• When attempting a supported opinion at the
  Superior level
• 94 do not address the task
• 73 resort to personal experience
• 77 lack precise vocabulary
• 88 unable to produce well-organized discourse

60
Russian findings Advanced Low and Mid

• When attempting to hypothesize at the Superior
  level
• 0 address the task
• 38 initiate but cannot complete task
• 27 lack specific vocabulary
• 33 marked by English interference
• 55 revert to describing a personal experience
• 55 are unable to produce well-organized extended
  discourse

61
Russian findings Advanced High

• When attempting to discuss a topic from an
  abstract perspective at the Superior level
• 71 initiate response but cannot complete
• 51 unable to deal with issue abstractly
• 42 revert to examples of personal experience
• 57 lack extended discourse
• 42 lack precise vocabulary

62
Russian findings Advanced High

• When attempting supported opinion at the Superior
  level
• 86 do not address the task
• 71 lack communication strategies
• 57 initiate but cannot complete the task
• 57 resort to personal experience
• 71 unable to deal with abstract
• 71 lack precise vocabulary
• 71 lack well-organized extended discourse

63
Russian findings Advanced High

• When attempting to hypothesize at the Superior
  level
• 85 refer primarily to American culture
• 42 do not address the task
• 42 lack communication strategies

64
Similarities across languages

• For both language groups, talking about a current
  event was the most challenging at the Advanced
  level
• These speakers tend to use language only in
  familiar, informal environments that do not move
  beyond the autobiographical

65
Similarities across languages

• For both language groups, sustaining the
  functions at the Superior level was the most
  challenging
• The ability to support opinion, deal abstractly,
  and hypothesize in cohesive and internally
  organized extended discourse

66
Similarities across languages

• Proficiency levels increased with
• More contact with heritage culture
• In both cases, Advanced and Superior groups
  either lived in a country where the heritage
  language is spoken or spent significant time
  there
• Use of heritage language
• The higher the proficiency level, the greater the
  use of heritage language or a mixture of heritage
  language and English
• Formal instruction in the heritage language at
  the college level

67
Implications for instruction

• Explicit and formal instruction in the language
  is critical for heritage speakers to reach full
  professional proficiency (Superior)
• Strong connection between those who had formal
  (college level) instruction in the language and
  those who reached higher proficiency levels
• Misperception that simply speaking the language
  at home and with friends is sufficient for the
  workplace likely linked to their tendency to
  over-assess their abilities and conclude that
  formal instruction is not necessary because they
  already speak the language

68
Implications for instruction

• Instruction should focus on
• Awareness of what is defined as Superior level
  language
• Functions, contexts and content areas, the text
  type, and the expectations for accuracy
• Expansion of contexts and content areas beyond
  personal and anecdotal
• Expansion of the lexical base to include precise
  (rather than generic) vocabulary
• Producing coherent extended discourse that goes
  beyond the single paragraph
• Dealing with topics from abstract perspective
  (issues)

69
Implications for OPI testing

• Heritage profiles differ from true L2 learner
  profiles
• Even at Intermediate, fluency and pronunciation
  may sound native-like
• Native-like pronunciation and lots of fluency do
  not compensate for lack of sustained functional
  ability
• Testers must keep the focus on the ability to
  address and complete the functions

70
Implications for assessments

• OPI remains a global functional assessment for
  ALL speakers
• Possible reporting of scores differently, special
  feedback form
• Purpose of assessment of heritage language
  learners
• Summative
• Formative
• Diagnostic

71
Next steps

• Continue to add the pools of samples
• Make samples available to researchers for further
  discourse analysis
• Publish results
• Expand project to other languages
• Mandarin
• Hindi

72
Questions?