TASK-BASED ASSESSMENT: INTEGRATING RESEARCH AND PROFESSIONAL DEVELOPMENT Geoff Brindley, Stephen Moore, Helen Slatyer

1
TASK-BASED ASSESSMENT INTEGRATING RESEARCH AND
PROFESSIONAL DEVELOPMENTGeoff Brindley, Stephen
Moore, Helen Slatyer Steven Ross Macquarie
University, Sydney, Australia

• International Conference on Task-Based Language
  Teaching
• Leuven 2005

2
Background The Certificates in Spoken and
Written English(CSWE)

• The Adult Migrant English Program (AMEP)
• Competency-based curriculum and assessment
  framework
• Explicitly described learning outcomes at four
  proficiency levels

3
CSWE assessment Validity and reliability issues

• Construct validity (cf listening/ reading
  descriptors can invisible operations be
  described?
• Comparability of teacher-developed tasks
• Consistency of rater behaviour
• Generalizability of skills across task types

4
Research findings

• Speaking
• -Effects of interlocutor behaviour on test scores
• -Effects of task type on difficulty (cf
  structured vs - unstructured)
• Listening
• -Effects of task characteristics on scores (cf
  speed, item format)
• Writing
• -Problems with interpretability of performance
  criteria
• -Rater halo effect
• Rater inconsistency

5
Research findings (contd)

• TASK DIFFICULTY TEACHER PERCEPTIONS OF FACTORS
  AFFECTING DIFFICULTY
• Lack of fit between task and text
• Task type design and cognitive load
• Ambiguity in question wording

6
Research findings (contd)

• LEARNER PERCEPTIONS OF FACTORS AFFECTING TASK
  DIFFICULTY
• Match between text and question
• Ambiguity in question wording
• Lexical knowledge

7
Creation of an assessment resource for teachers

• Aims
• To develop a bank of trialled assessment tasks to
  be made available on-line
• To create a program of professional development
  in assessment task design

8
Creation of an assessment resource (contd)

• Task development process
• Tasks sourced from teachers or new tasks
  developed by NWG
• Tasks vetted, modified and piloted by NWG
• Tasks moderated by panel of teachers and
  academics
• Tasks trialled nationally

9
Addressing professional development needs

• Professional development program derived from
  task development workshops
• Devolvement of responsibility to teachers and
  educational institutions
• Workshops provide a forum for exchange
• It has been extremely useful and rewarding to be
  involved in the national think-tank
• (one of the most valuable aspects is) the
  opportunity to meet people round the country

10
Future developments

• Task bank
• Improve the quality of tasks
• Increase the number of tasks
• Extend coverage of LOs to acccommodate curriculum
  changes

11
Future developments (contd)

• Professional development program
• Online PD kit resources for local PD
• -Choice of 3 workshops
• 1. Task design issues
• 2. Task quality evaluation
• 3. Development of new tasks

12
Future developments (contd)

• Professional development program
• PD kit includes Powerpoint presentations,
  assessment materials and readings
• Develop workshop formats according to needs
• Develop interactive self-access package

13
Future developments (contd)

• Research
• Refine think aloud protocol analysis
• Investigate effect of specific task/item types on
  perceptions of difficulty

14
Conclusions

• Four key principles underpin project
• Partnership between educational institutions,
  teachers and university research centre
• Devolvement of responsibility to institutions
  teachers through workshop program
• Sustainability of the project through a
  commitment from stakeholders in government and
  educational institutions
• Ongoing program of collaborative research