Good Practices in Assessment and Measurement of Learning Outcomes of Academic Programmes

1
Good Practices in Assessment and Measurement of
Learning Outcomes of Academic Programmes
Lingnan UniversityMay 25, 2009
David M. Kennedy, PhD ??? ??
2
Outline

• Short review of outcomes-based approaches to TL
• Learning design
• Assessment
• Measurement of learning outcomes
• Albert Einstein
• Not everything that can be counted counts, and
  not everything that counts can be counted

3
What is OBA ??
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
Biggs, J. B. (2003). Teaching for quality
learning at university (2nd ed.). Buckingham
Open University Press.
9
Graduate attributes in the 21st century
http//www.itl.usyd.edu.au/graduateattributes/imag
es/triangle.gif
10
(No Transcript)
11
Learning designs I

• Learners
• experience, expectations, learning style and
  beliefs
• Teachers
• capabilities, goals, experience and beliefs
• Learning resources
• physical and virtual
• Discipline and industry
• knowledge and practice
• Scholarship
• evaluation of innovation, publishing and practice

12
Assessment
13
(No Transcript)
14
SOLO Taxonomy Biggs Collis

• Structure of the Observed Learning Outcomes
• A method to categorise students responses to
  open-ended questions
• Focus on qualitative differences between
  students responses

15
Applicability

• Course aims
• Setting assessment
• Marking assessment
• Structuring your own writing

16
Five SOLO levels

• Missing the point
• Single point
• Multiple unrelated points
• Logically related answer
• Unanticipated extension

17
See SOLOTaxonomyVerbs.doc
18
SOLO in use

• Assessing (teachers)
• forum postings by students (not word counts)
• concept or mind maps by students
• Assessing (students)
• peer assessment of forums (for critical thinking,
  evaluation and analysis)

19
Learning outcomes - evidence

• The old essay that is text-based only
• The new include new literacies
• Text including supporting literature
• Concept maps and mindmaps
• Electronic artifacts
• audio, video, posters
• Images (photos)

20
The new
21
(No Transcript)
22
Linking

• and

23
we should articulate what we expect our
students to learn, and we must gather evidence to
determine whether they have learned it. (Stone,
2005)
24
Multi-level approach
External
University
Programme
Course
25
Defining outcomes getting evidence of success

• How do we get evidence at each level?
• External expectations?
• Universitys Strategic Plan?
• Programme outcomes?
• Course outcomes?

26
Defining outcomes getting evidence of success

• How do we get evidence at each level?
• External expectations?
• Council, advisory committees
• Universitys Strategic Plan?
• Link to explicit policy, internal consultations
• Programme outcomes?
• Surveys, alumni, employers
• Course outcomes?
• Surveys, assessment data

27
(No Transcript)
28
Feedback for evaluation I

• Programme and course design
• Potential data
• student evaluation of teaching
• programme-level surveys
• organisational feedback (student placement)
• external examiner reports
• internal reviews
• student association feedback
• evaluation of TL innovation (TDGs)
• first year experience surveys

29
Feedback for evaluation II

• Assessment
• Potential data
• student examination and test data
• evidence of variety of assessment (to achieve
  graduate outcomes)
• learning outcomes x course matrices
• assessment/ outcomes in not-for-credit courses

30
Feedback for evaluation III

• Actual learning outcomes
• Potential data
• language competency data
• capstone and/ or final year projects
• programme-level data
• graduate and alumni surveys
• employer
• professional associations
• alumni (five-year) feedback

31
Monitoring over time

• Organisational infrastructure
• Evidence gathering needs to be efficient
• Tracking alumuni
• Technology can support evidence gathering and
  analysis
• e.g., efficient database management
• alumni email services or gmail?

http//www.wickedreport.com/images/Coolest-Clocks0
6.jpg
32
Questions

• Contact
• dkennedy_at_hku.hk
• davidmkennedy_at_LN.edu.hk

33
References

• Biggs, J. B. (2003). Teaching for quality
  learning at university (2nd ed.). Buckingham
  Open University Press.
• McNaught, C. (in press). More than a paper trail
  Developing quality assurance processes that
  enhance teaching and support student learning.
  Proceedings of the International Symposium on
  Development of Teachers Potentialities, Capital
  University of Economics and Business, Beijing,
  10-11 July 2009
• Stone, M. (2005). Opening remarks. Symposium on
  outcome-based approach to teaching, learning and
  assessment in higher education International
  perspectives. Retrieved May 15 2009 from
  http//www.ugc.edu.hk/eng/ugc/publication/speech/2
  005/sp171205.htm

34
Abstract

• Good Practices in the Assessment and Measurement
  of Learning Outcomes of Academic Programmes
• An outcomes-based approach (OBA) to teaching and
  learning involves three phases 1) clear
  articulation of desired learning outcomes, 2)
  good curriculum design whereby content, learning
  activities and assessment are chosen so that the
  student learning experience is likely to lead to
  the attainment of the desired learning outcomes,
  and 3) collecting evidence that students have
  attained the learning outcomes.
• This seminar will focus on the third area.
  Evidence of student learning includes careful
  analysis of assessment data, tracking students
  perceptions about their learning and a considered
  plan for monitoring what happens post-graduation
  including employer and alumni data.