Melissa Fincher

1
Lessons from the Field Computer-Based Testing
the Good, the Bad, and the Ugly
Georgias Experience with Online Assessment
Melissa Fincher mfincher_at_doe.k12.ga.us
2
Georgias Experience

• Online assessments in two arenas
• Formative assessments
• Online Assessment System (OAS)
• High stakes assessments
• End of Course Test (EOCT)

3
Online Assessment System

• Item bank available to teachers and students for
  formative use
• Began in early spring 2002
• Grades 1 8 and high school content
• Teacher and Student levels

4
EOCT

• Eight core high school courses
• Score contributes 15 of students course grade
• Online or paper and pencil
• Began in Winter 03

5
Overall Considerations

• Online assessment is great but there are trade
  offs for the benefits it provides
• Less paper to manage, but significant management
  is still required
• Capacity issues may require more flexibility in
  assessment procedures

6
Practical Considerations

• Capacity
• Logistics
• Partnerships
• Assumes comparability has been established

7
Capacity

• Consider capacity of the state to support the
  initiative
• Expect to provide considerable support to local
  systems and schools
• Consider the capacity of your vendor
• What other states do they support?
• What states have the same testing window as you?
• What is the procedure for handling/escalating
  problems?
• Is the Help Desk helpful? Do they help even
  though the issue may be local?
• How have they considered the needs of special
  population students?

8
Capacity

• Consider capacity of local systems
• Do they have enough equipment to handle the
  testing load?
• Do they have a coherent plan to cycle students
  through testing?
• Do they have the training to know what to do if
  they happen differently than expected?

9
Logistics

• Consider how to grow capacity at all levels
• Consider training needs at all levels
• Consider how the testing schedule will impact the
  instructional day
• Consider test security its different in a
  computer lab setting

10
Logistics

• Consider backup plansexpect the unexpected and
  be prepared
• Require contractor to observe testing at schools
  often
• Require contractor to monitor calls and keep
  records review these records
• Early detection of issues is critical
• Whats the escalation procedure???

11
Partnerships

• Testing should partner with Technology to develop
  RFP and evaluate
• Not solely a technology initiative nor is it
  solely a testing initiative
• Negotiate ownershipwork towards a collaborative
  relationship
• Encourage local System Testing Coordinators
  should partner with local Technology Coordinators

12
Conclusion

• When things go right, its fabulous but
• when they dont, its something else be
  prepared!
• States should always have a back up plan (or two).