e-Assessment

1
e-Assessment
Key Issues March 2009
2
Formative Assessment

• Key Achievements10 fully worked up patterns,
  9 casesNew formative assessment definition
  with underpinning key aspectsApplication of
  proven methodology to bridge chasm between
  pedagogues and technical developers
• Key FindingsPatterns need transferability
  proofPatterns need prioritising against criteria
  (national/institutional drivers)Software needs
  building to demonstrate patterns (prove method
  too)Open Mentor pattern has guerrilla payload
  (feedback on feedback) potentially powerful
• Key Recommendations for further workEASI Project
  (Southampton) to trial outputs through
  application / extension of formative assessment
  repositoryIf feedback isnt working its not
  formative exploreFeedback plus encouraging
  learner self reflection / regulationRole of
  comment banks in enabling construction of good
  feedbackMore patterns to support more pain
  situations in curricula

3
Advanced e-Assessment Techniques

• Key AchievementsCatalogue of techniques6 in
  depth case studiesAdaptive testing
  (Auckland)Combining human/computer marking
  (Manchester)Higher order skills (Federation of
  Medical Boards USA)Language testing (Kent,
  Pearson)Short text compendium (Automark, Dundee,
  IAT, OU)Storage / Item Banks
• Key FindingsTechniques not well used (yet) in
  HE/FEInnovations are isolated (rely on an
  enthusiast) scale up issuesLack of adoption of
  technical standardsHuman concerns of using ICT
  in assessment
• Key Recommendations for further workBenefits of
  e-Assessment techniques must be more widely
  understoodRetro fit standards? But do standards
  stifle innovation? Are the standards fit for
  purpose?Opening up institutional repositories
  for discovery of such initiatives

4
e-Assessment in Wales

• Key AchievementsDiscipline based assessments
  generatedProject has addressed key cross sector
  collaborative barriers
• Key FindingsCultural/technological divide across
  member colleges and lead are barriersDiagnostic
  assessment helps retentionVideo in summative
  assessments powerfulBridging from FE to HE
  study/key skills
• Key Recommendations for further workCrack
  cultural/technological for HE/FE (and FE/HE)
  divide barrierseg e-Assessment time doesnt
  count as contact hours in FE

5
Quality in e-Assessment

• Key AchievementsIdentified likely quality issues
  (psychometrics etc)Looked across HE, commerce,
  other sectors
• Key FindingsLots of delivery issues
  discoveredLittle on validity, reliability etc
  and some inappropriate useNo one mentioned the
  relevant BSsThis isnt done in traditional
  assessment eitherContrast between US and UK
  insurance against litigation
• Key Recommendations for further workGuidance for
  test providers to substantiate evidence to
  address quality for all types of
  assessmentAwareness raising is keyTool kit for
  lightweight assistanceSell on the back of item
  bankingJoin up with Pathfinder initiative

6
e-Portfolio for assessment

• Key AchievementsLandscape scan, case studies of
  practice
• Key FindingsApproach limited to vocational
  coursesDrivers were convenience, not
  qualityStudents like approach but reluctant to
  share information beyond tutorBarriers include
  institutional policies, interoperability of
  contentReluctance to adapt practice to exploit
  the tools (evidence selection)Impoverishment due
  to lack of customisationTutor preference to
  assess process or productLittle evidence for
  genuine transformation of pedagogical practice
• Key Recommendations for further workAll in the
  report!Discipline specific portfolio needs vs
  institutionally supported toolsCurriculum
  designed to exploit portfolio