Empowering Teachers as Testers: Developing Assessment Literacy

1
Empowering Teachers as TestersDeveloping
Assessment Literacy

• Christine Coombe
• Dubai Mens College

2
(No Transcript)
3
Presentation Agenda

• My background testing philosophy
• Views about assessment
• Changing assessment landscape
• Models of in-house test development
• Challenges threats to teacher involvement
• Techniques for developing teachers as testers
• Questions Comments

4
My Philosophy of Testing

• Testing means assessment in a broad sense
• It encompasses all types of assessment
• Always guided by
• cornerstones of good testing practice
• seven steps to fair assessment (Code of Fair
  Testing Practices in Education)
• Involvement in assessment is every Ts
  responsibility!
• Educational reform will not be productive until
  Ts master the basic principles of sound
  assessment
• Testing is one of most misunderstood areas in ELT

5
How Students View Assessment

• Generally seen as something done to students by
  teachers
• Ss see tests as threats to their competence
• Ss may feel panic and confusion
• see tests as something to be got through
• more able Ss enjoy the experience
• most Ss feel anxious, worried and inadequate
• when tests are high-stakes, Ss suffer from test
  anxiety
• There is great pressure on Ss to succeed
• if they dont, they are branded as failures

6
How Teachers View Assessment

• Similar feelings to those of Ss
• Ts far removed from development process
• Tests are the only useful way of motivating Ss
• View that assessment occurs after learning has
  finished
• Gap between teaching testing
• testers not in touch with classroom realities
• Least fun area for Ts (Jacob Chase, 1992)

7
How Education Boards View Assessment

• Almost all major educational boards have
  identified endorsed a set of assessment
  competencies for Ts
• National Council on Measurement in Education
  (NCME)
• National Education Association
• American Federation of Teachers
• Despite universal buy in, assessment literacy is
  still not a prerequisite for hiring in majority
  of states

8
What is an Assessment-literate Teacher?

• A teacher who knows
• the difference between sound unsound assessment
• how to meet specific standards of quality
• the characteristics of sound assessment
• Arise from serve clear purposes
• Arise from reflect clear/appropriate
  achievement targets
• Utilize an appropriate assessment method
• Sample student achievement appropriately
• Control for all relevant sources of bias
  distortion

9
Characteristics of Assessment-literate Teachers

• Assessment-literate teachers come to any
  assessment knowing
• what they are assessing
• why they are assessing it
• how best to assess the proficiency/content
• how to generate sound samples of performance
• what can go wrong
• how best to prevent problems before they occur

10
Research on Assessment Knowledge

• A handful of studies have investigated this
  important area
• Teacher education programs do not offer Ts
  sufficient training in assessment (Rodgers, 1991)
  
• Research shows that student assessment is one of
  the Ts most demanding, complex and important
  jobs (Shulman, 1986 Calderhead, 1996)
• Survey of 121 applied linguists about their
  knowledge of and attitudes towards statistics and
  empirical research (Lazarton, Riggenbach
  Ediger, 1987)
• Survey of practicum component of TESOL MA holders
  in US Japan (Richards Crookes, 1988 Richards
  Hino, 1983)
• An understanding gap was found to exist (Bailey
  Brown, 1996)

11
An Understanding Gap in Language Testing

• Jones (198516) pointed out
• there is really a sub-profession within language
  teaching that is dedicated to the science of
  language proficiency measurement. The level of
  sophistication has become so great that an
  understanding gap has emerged between the testing
  specialists and their colleagues in other areas
  of language teaching.

12
The Changing Role of Teachers

• Traditionally, testing left to specialists
• Testing is too important to be left in the hands
  of testers! (Alderson, 1999)
• inappropriate even damaging
• testers cannot work alone
• we depend on expertise, experience cooperation
  of Ts
• Aim involve teachers in all aspects of testing
• Promote transparency for Ts and Ss
• Make feedback accessible, meaningful

13
Why Involve Teachers?

• Assessment and feedback are central to teaching
  (Alderson, 1999)
• Who knows best about whats happening in the
  classroom?
• Ts needed so that tests are reflective of the
  curriculum and applicable to Ss levels
• tests must be close to curriculum teaching
• no correspondence, test does not reflect best
  practice
• teaching should also be close to tests
• if tests are good Ts dont need to change what
  they do
• test preparation books are just good textbooks

14
Further Rationale for Teacher Involvement

• Ts must feel ownership of a test
• 20-30 of our time as Ts is spent in
  assessment/assessment-related activities
• Involvement in assessment is every Ts
  responsibility!
• Minimal Ts understand test purpose, nature
  philosophy
• Ideal Above plus participation in test
  construction, piloting, reporting etc.

15
Why is Testing left to Testers?

• Reasons Ts dont become involved
• Fear (Stiggins, 1995)
• Accumulation of layers of negative emotions
  associated with assessment
• Ivory Tower belief Field is arcane (Alderson,
  1999)
• journals not accessible to the classroom teacher
• Inadequate or poor training
• Concerns close to Ts daily lives
• feelings of insecurity or lack of confidence

16
Why is Testing left to Testers?

• More reasons why Ts dont become involved
• Insufficient time to assess well
• Insufficient resources allocated to assessment
• administrators pay lip service to importance of
  assessment no release time or extra remuneration
• Its easier not to have to worry about assessment
• All these factors conspire against teacher
  involvement!

17
Four Models of In-house Test Development

• Teachers write the tests.
• Testing specialists write the tests.
• Administration buys the test.
• Teachers and testers write the tests
  collaboratively. (Alderson, Clapham Wall, 1995
  as cited in Marsden, 1999)

18
Model 1 Teachers Write the Tests

• Advantages
• Reduces gap between teaching and testing
• Teachers feel ownership for the test
• Administration feels it is saving money
• Disadvantages
• Tests are rarely reliable because of lack of
  training and expertise amongst teachers
• Leads to one time use of test
• Waste of valuable resources
• Workload so heavy that Ts bow out of process

19
Model 2 Testing Specialists Write the Tests

• Advantages
• Increased reliability
• specialists involved, statistics generated
• Decreased teacher workload
• Removes Ts from responsibility
• Disadvantages
• Gap between teaching/learning
• Ts feel alienated and left out
• Most importantly, Ts dont have opportunity to
  develop/improve test writing skills

20
Model 3 Administration Buys the Test

• Advantages
• High status and reliability if test is
  internationally recognized
• Saves time
• No arguments about results
• Disadvantages
• Potential mismatch between teaching/ learning
• Costly
• Ts dont have opportunity to improve test
  development/writing skills

21
Model 4 Teachers/Testers Write the Tests

• Advantages
• Testers can share expertise with teachers
• Teachers can give classroom insights to testers
• Results in greater buy-in from Ts
• Reduces teaching/testing gap
• Seen by all to be an efficient use of resources
• Disadvantages
• Long lead in time
• Must start months in advance
• Needs administrative commitment for success
• Release time, extra remuneration PD support

22
Assessment Skills Required by Teachers

• In order to ensure valid and reliable tests, Ts
  need a wide range of skills (Brindley, 2000)
• Observing, interpreting documenting Ss
  language use
• designing classroom tests assessment tasks
• analyzing test results
• providing diagnostic feedback to Ss
• evaluating the quality of tests tasks
• evaluating learner performance according to
  rating scales
• writing evaluative reports

23
Seven Standards for Teacher Development in
Assessment

• Developed by American Federation of Teachers, the
  National Council on Measurement in Education and
  the National Education Association (1990)
• Teachers should be skilled in
• Choosing assessment methods appropriate for
  instructional decisions
• Developing appropriate assessment methods
• Administering, scoring, and interpreting the
  results of both externally produced and
  teacher-produced assessment methods

24
Seven Standards Cont

• Teachers should be skilled in
• Using assessment results when making decisions
  about individual students, planning teaching,
  developing curriculum and improving schools
• Developing valid grading procedures which use
  student assessment
• Communicating assessment results to students,
  parents, and other stakeholders
• Recognizing unethical, illegal inappropriate
  assessment methods and uses of assessment
  information

25
Continued Threats to Teacher Involvement

• No time to pursue much needed PD
• developing expertise requires lots of hands-on
  experience, additional training/coursework
  formal degree programs
• Insufficient resources
• No money to reduce teacher workload
• Trust issues
• feeling of whether Ts can be trusted to write
  tests or contribute to test development
• their Ss might have an unfair advantage

26
The Challenges

• Getting institutional or administrative support
• No idea how time consuming assessment can be
• Overcoming increased workloads
• Testing baggage that Ts bring from their own
  educational background or institutional context
• Getting Ts to plan the whole cycle of test
  development
• Program/curricular problems
• mismatch between teaching and testing
• Slippery slopes of alternative assessment

27
Techniques for Developing Teachers as Testers

• Variety of resources for development
• Importance of hands-on experience in supportive
  environment
• Professional Development
• Both theoretical practical sessions
• HCT five-day Assessment PD course
• Print resources
• A Practical Guide to Assessing English Language
  Learners, Coombe, Folse Hubley, 2007 University
  of Michigan Press.

28
Electronic Resources

• We are not remote now!
• Effective use of email
• drafting, critiquing, sharing
• mentoring when most needed
• Internet resources
• Glenn Fulchers Resources in Language Testing
  website http//www.le.ac.uk/education/testing/ltr
  .html
• ERIC resources

29
Where do we go from here?

• Identify common issues for development, research
  and further PD
• Use technology to develop online workshops and
  support groups
• Keep in touch..help is an email away!
• Other suggestions?

30
Questions Comments

• Presenter contact
• Christine Coombe
• Dubai Mens College, HCT
• PO Box 15825
• Dubai, United Arab Emirates christine.coombe_at_hct.a
  c.ae