RELTA: Design and Delivery

1
RELTA Design and Delivery
2
RMIT University

• RMIT Royal Melbourne Institute of Technology
• Melbourne, Australia
• Technology main area founded 1890s, now a
  university

3
ICAO LPRs and Testing

• Annex 1
• Pilots and ATCOs
• Speak and understand language for radiotelephony
  communications
• Assessed from Levels 1-6
• Appendix
• Language proficiency requirements applicable to
  the use of both phraseologies and plain language
• Holistic Descriptors
• Communicate in voice-only (radiotelephone) and in
  face-to-face situations
• Communicate on common, concrete and work related
  topics
• Resolve misunderstandings in general or
  work-related context
• Communicate effectively when presented with
  complications/unexpected events
• Use an intelligible dialect/accent

• Annex 1
• Pilots and ATCOs
• Speak and understand language for radiotelephony
  communications
• Assessed from Levels 1-6
• Appendix
• Language proficiency requirements applicable to
  the use of both phraseologies and plain language
• Holistic Descriptors
• Communicate in voice-only (radiotelephone) and in
  face-to-face situations
• Communicate on common, concrete and work related
  topics
• Resolve misunderstandings in general or
  work-related context
• Communicate effectively when presented with
  complications/unexpected events
• Use an intelligible dialect/accent

• speak understand radiotelephony
• phraseologies
• plain language
• voice-only
• face-to-face
• work related
• resolve misunderstandings
• complications/unexpected events
• dialect/accent

4
ICAO LPRs and Testing

• Validity
• Reliability
• Authenticity
• Interactivity
• Fairness
• Practicality

• ICAO LPRs
• Radiotelephony
• Phraseology
• Plain English
• Routine?non-routine
• Voice-only
• Face-to-face
• 6 ICAO criteria
• 6 bands of ICAO scale

ICAO-compliant tests
5
Aviation Language Construct

• Phraseology radiotelephony
• Natural English radiotelephony
• Natural English face-to-face

6
Phraseology

Codified language situational routines
functional routines
7
Natural English in RT

Plain English in radiotelephony
Pilot Pacific 183, we have a further problem
with our leading edge flap extension and believe
this is due to the prior icing issue perhaps
just a micro-switch malfunction, however well be
conducting a flapless landing and will require
runway 26 Left or Right. ATC Pacific 183 roger,
change of Runway 26 Left, turn right heading
080 for downwind, now with 24 track miles to
run, amend descent, maintain 4,000.
Pilot Pacific 183, we have a further problem
with our leading edge flap extension and believe
this is due to the prior icing issue perhaps
just a micro-switch malfunction, however well be
conducting a flapless landing and will require
runway 26 Left or Right. ATC Pacific 183 roger,
change of Runway 26 Left, turn right heading
080 for downwind, now with 24 track miles to
run, amend descent, maintain 4,000.
8
RELTA Design Rationale

• Design rationale assess language in
  aviation-specific TLU situations
• Maximise opportunity for engagement with
  test-tasks (authenticity)
• Minimise construct irrelevant variance
• Assess Speaking and Listening competence
  (separately)
• Assess language of pilots and ATCOs via
  different test forms
• Designed in response to ICAO LPRs

9
Skills and Domains
10
Target users
11
Speaking Section 1

Section Mode of delivery Language assessed
Section 1 Voice only Role-play Computer-mediated Interlocutor controls visual/audible input Phraseology in routine radiotelephony communications
12
Speaking Section 2

Section Mode of delivery Language assessed
Section 2 Voice only Role-play Computer-mediated Interlocutor controls visual/audible input Plain English in complex non-routine radiotelephony communications
13
Examiner script

14
Speaking Section 3

Section Mode of delivery Language assessed
Section 3 Face-to-face Interlocutor asks prescribed questions (contained in Examiner booklet). Plain English in aviation-specific conversation
15
Examiner script

16
Speaking Light Aircraft
17
Speaking ATC TWR
18
Speaking ATC ENR
19
Speaking ATC APP
20
Listening Section 1

21
Design Section 2

22
Design Section 3

23
RELTA Test Construct

24
Test Development

Needs analysis
ICAO LPRs
Specifications
SMEs
Target users
Prototype
Piloting
Target users
Refinement
SMEs
Trialling
Versions
Test bank
Monitoring
Reviewing
25
Construct validation

• Does the test assess language proficiency?

26
Construct validation

• How do test components interact?
• Does each test component serve a function?
• Statistical tools
• correlations between sections, listening and
  speaking, with other tests
• mean scores compare sections and difficulty

27
Best Practice Standards

• Validity
• Reliability
• Authenticity
• Interactivity
• Fairness
• Practicality

28

• michael.kay_at_rmit.edu.au
• www.RELTA.org