Developing objectives and learning outcomes for the evaluation of learning

1
Developing objectives and learning outcomes for
the evaluation of learning

• InfoSkills2 FOLIO course

2
Introduction

• In the last briefing, Evaluating learning
  (http//www.nelh.nhs.uk/folio/infoskills2/learning
  .htm), we saw that one way to evaluate learning
  is by setting assignments for trainees
• To aid the evaluation of learning, it is useful
  to match the assessment criteria by which you
  will mark assignments with the objectives and
  learning outcomes of the training course
• Objectives, learning outcomes and assessment
  criteria should be created during the design of
  information skills training courses. For more
  information about designing training courses, see
  the Designing information skills training
  courses briefing from the previous InfoSkills
  course at http//www.nelh.nhs.uk/folio/infoskills
  /methods.htm

3
Objectives and learning outcomes

• So what are objectives and learning outcomes
  and how do we create them?

4
What are objectives?

• Objectives are
• An expression in concrete and specific terms of
  what will happen on the course. The specific
  activities which participants will undertake on
  the course in order to achieve the stated aims
  (1).
• Note not what will be learned, that comes
  next, but what will be done (1).
• Objectives usually start with the phrase By the
  end of this training course, participants will be
  familiar with (or similar)

5
Examples of objectives

• For a literature searching course (2)
• By the end of this session, participants will be
  familiar with
• the names, coverage and content of the core
  health and health-related databases including
  Medline, CINAHL and Embase
• the types of information that can be identified
  through interrogation of these databases
• the process of developing a research question
  into an effective search strategy including the
  identification of keywords, concepts and synonyms
• the techniques and skills required to conduct a
  database search including the use of a controlled
  language thesaurus, truncation, boolean operators
  and explosion

6
Examples of objectives

• For a searching for health information on the
  internet course (2)
• By the end of this session, participants will be
  familiar with
• the various components of the Internet, such as
  electronic mail and the World Wide Web
• the basic structure of, and methods of
  navigating, the World Wide Web
• the types of information which are available
  through the internet
• the strengths and weaknesses of the Internet as
  an information resource
• criteria that can be used to evaluate the quality
  of internet-based information
• examples of resources which can be used to find
  evidence to support research and practice
• the techniques and skills needed to find
  resources on the internet including the use of
  information gateways, search engines and resource
  lists

7
Examples of objectives

• For a library induction
• By the end of this session, participants will be
  familiar with
• the opening hours of the library
• the resources held by the library (books,
  journals, statistics etc) and how to locate them
  (including an introduction to the NLM
  classification scheme)
• their borrowing rights
• the services offered by the library (literature
  searching training, inter-library loans, enquiry
  desk etc)
• the contact details of the library (including
  email address, telephone number, web address)

8
What are learning outcomes?

• Learning outcomes (also known as key learning
  activities) are
• What will be learned. What the participants
  will be expected to achieve at the end of the
  course - knowledge, skills, and attitudes.
  Specifically what achieving the course objectives
  should have enabled the participants to learn (1)

9
Writing learning outcomes (1)

• Learning outcomes should specify the minimum
  acceptable standards for students to pass the
  training course (3)
• Therefore, it is important that learning outcomes
  are expressed in terms of the essential learning
  for the information skills training course (3)
• Each information skills training course should
  have between four and eight key learning outcomes
  (3)

10
Writing learning outcomes (2)

• Learning outcomes should begin with the following
  phrase (or similar) (3)
• On successful completion of this training
  course, students will be able to

11
Writing learning outcomes (3)

• Each learning outcome should be preceded by an
  action verb so that students are able to
  demonstrate that they have learned or achieved
  the outcome (3)
• Verbs relating to knowledge outcomes such as-
  know, understand, appreciate- tend to be
  rather vague and focus on the process that
  students have gone through (3)
• Instead, where possible, action verbs such as-
  solve, evaluate, use, analyse- should be
  used to indicate how students can demonstrate
  their knowledge (3)

12
Writing learning outcomes (4)

• Blooms Taxonomy of 1956 is a good aid for
  writing learning outcomes (3)
• Bloom identified six categories of learning (3)
• 1. Knowledge 2. Comprehension 3. Application 4.
  Analysis 5. Synthesis 6. Evaluation
• The first two relate to knowledge and
  understanding, while the remaining for involve
  intellectual skills
• It may be tempting to concentrate on the first
  two categories for your information skills
  training session, but it is important to
  introduce the higher level activities where
  possible

13
Examples of learning outcomes Stage 1

• For an information skills for medical students
  training course
• Upon successful completion of this training
  course, participants will be able to

14
Examples of learning outcomes Stage 2

• For an information skills for medical students
  training course
• Upon successful completion of this training
  course, participants will be able to
• Understand
• Develop
• Conduct
• Evaluate and select
• Use
• Use
• Use
• Understand and evaluate

15
Examples of learning outcomes Stage 3

• For an information skills for medical students
  training course Based on (4)
• Upon successful completion of this training
  course, participants will be able to
• Understand the information seeking process and
  its role in research
• Develop a search strategy for your research
  topic(s)
• Conduct subject searches and other search tactics
• Evaluate and select information
• Use traditional library sources
• Use on-line databases
• Use the WWW for healthcare/medical information
• Understand and evaluate the range of primary
  healthcare resources available to you

16
What are the benefits of learning outcomes?

• Courses which are designed using learning
  outcomes are student-centred (2)
• Designing courses in this way marks a shift from
  the content of a module or course (i.e. what
  trainers teach) towards its outcome (i.e. what
  the students are able to do upon successful
  completion of the course) (2)

17
What are the benefits of learning outcomes?

• Learning outcomes can
• Help to guide students in their learning because
  they explain what is expected of them (3).
• Help trainers to focus on what they want students
  to achieve as a result of the training course
  provide a useful guide to inform potential
  students and employers about the knowledge and
  understanding that a graduate of the training
  course will possess (3)

18
What about assessment criteria?

• Now that we know what objectives and outcomes
  are, we need an understanding of assessment
  criteria

19
Assessment criteria

• Assessment criteria describe how well a student
  has to be able to achieve the learning outcome in
  order to be awarded a particular grade (5).
• Assessment criteria should test, assess or
  relate to the learning that is mentioned in the
  learning outcomes (6)
• Assessment criteria are important because they
  ensure that marking is fair and standard as all
  students work is marked to the same criteria (5)
• For an example of assessment criteria for an
  information literacy course, see
  http//www.nelh.nhs.uk/folio/infoskills2/criteria.
  doc
• For detailed information about how to create
  assessment criteria, see Designing and using
  assessment criteria (Oxford Brookes University)
  at http//www.brookes.ac.uk/services/ocsd/firstwo
  rds/fw24.html

20
Linking objectives, learning outcomes and
assessment criteria to the evaluation of learning

• When setting assignments or tests for trainees,
  the tasks that you set should have a clear
  alignment with the objectives, learning outcomes
  and the assessment criteria for the course (3)
• This enables you to design appropriate tasks and
  to deliver the training course in a way that
  enables students to reach the required outcomes
  (3)
• It also enables you to ensure that there is
  coherence in your information skills training
  session (3)
• The alignment between objectives, learning
  outcomes, assessment tasks and assessment
  criteria should be made transparent to the
  students to help them to successfully attain the
  required knowledge and skills (3)

21
References

• West Midlands Modular Trainers Course. 2004.
  Curriculum. http//www.trainer.org.uk/members/theo
  ry/planning/curriculum.htm
• Alison Hicks. Developing information skills
  training for National Health Service personnel
  experiences at the Trent Institute for Health
  Services Research. Program, 1998. Vol 32, No. 2
  pp 123-136
• University of Central England in Birmingham.
  2004. Guide to learning outcomes.
  http//lmu.uce.ac.uk/outcomes/UCE20Guide20to20L
  earning20Outcomes.pdf

22
References

• University of Strathclyde. 2004. GAELS.
  http//gaels.lib.strath.ac.uk/info_skills
• Oxford Brookes University. 2001. 2.4 Designing
  and using assessment criteria. http//www.brookes.
  ac.uk/services/ocsd/firstwords/fw24.html
• Jenny Moon. Linking Levels. Learning Outcomes and
  Assessment Criteria. http//www.liv.ac.uk/cll/fil
  es/LinkingLevelsplusasscrit.doc Exeter University

23
Further reading

• Chris Butcher. 2004. Blooms Taxonomy. SDDU
  University of Leeds. http//www.leeds.ac.uk/sddu/o
  nline/bloom.htm
• E.A. Hesketh. 2004. Aims and objectives. NHS
  Education for Scotland. http//www.nes.scot.nhs.uk
  /courses/ti/AimsObjectives.pdf
• Janet Peters. 2004.Learning outcomes and
  information literacy. SCONUL
• University of Aberdeen. 1997. Guidance on aims
  and objectives for teaching and learning.
• http//www.abdn.ac.uk/secretariat/aimsobs.hti
• University of Portsmouth. 2004. Aims and
  objectives explained getting the most out of
  units. http//www.port.ac.uk/departments/academic/
  edam/resources/intranet/studyguide/filetodownload,
  3626,en.pdf