How to make sure your Students Learn what you want them to

1
How to make sure your Students Learn what you
want them to
Claus Brabrand ((( brabrand_at_itu.dk ))) (((
http//www.itu.dk/people/brabrand/ ))) Associate
Professor, Ph.D.Software and Systems Section
IT University of Copenhagen, Denmark
2
An Introduction toConstructive Alignment and
The SOLO Taxonomy
Claus Brabrand ((( brabrand_at_itu.dk ))) (((
http//www.itu.dk/people/brabrand/ ))) Associate
Professor, Ph.D.Software and Systems Section
IT University of Copenhagen, Denmark
3
Exercise
T
"What is good teaching?"
4
Outline (Oct 10, 2013)

• 1) Introduction
• Constructive Alignment
• The SOLO Taxonomy
• 2) From Content to Competence
• Advocate a shift in perspective
• Elaborate on The SOLO Taxonomy
• 3) In Practice
• Concrete recommendations
• Alignment implementation process

5
Introduction to

• Constructive Alignment SOLO Taxonomy

John Biggs popular and heavily cited book
Teaching for Quality Learning at
University - What the student does
J. Biggs C. Tang, 2007
Teaching Teaching
Understanding Understanding
19 min award-winning short-film on Constructive
Alignment (available on DVD in 7 languages,
epilogue by John Biggs)
6
Neighbour Discussion
T
"What are the film'smain messages (in your
opinion)"?
7
Outline (Oct 10, 2013)

• 1) Introduction
• Constructive Alignment
• The SOLO Taxonomy
• 2) From Content to Competence
• Advocate a shift in perspective
• Elaborate on The SOLO Taxonomy
• 3) In Practice
• Concrete recommendations
• Alignment implementation process

8
From Content to Competence

• My old course descriptions (Concurrency 2004)
• Given in terms of a 'content description'.
• Essentially

• Goal is
• To understand
• deadlock
• interference
• synchronization
• ...

This is a bad idea for two reasons...!
9
Problem 1 !

• Problem with 'content' as goals !

analyze ... theorize ...
analyze systems explain causes
define deadlock describe solutions
agreement
Stud. C

• Goal is
• To understand
• deadlock
• interference
• synchronization
• ...

tacit knowledge from a research-based tradition
not known by student
Teacher
name solutions recite conditons
analyze systems explain causes
Stud. B
?
Stud. A
Censor
10
Problem 2 !

• Problem with 'understanding' as goals !

• Goal is
• To understand
• deadlock
• interference
• synchronization
• ...

'concept of deadlock' ?!
?
The answer is simple
It cannot be measured !
11
Competence !

• 'Competence' as goals !

Competence knowledge capacity to
act upon it
Have the student do something and then observe
(evaluate) the product and/or process

• Objective !
• To learn how to
• analyze systems for...
• explain cause/effects...
• prove properties of...
• compare methods of...
• ...

Note 'understanding' is of course
pre-requisitional !
?
Note' inherently operational ( verbs)
'SOLO' Structure of the Observed Learning
Outcome
12
Neighbour Discussion
T
"How does this content vs competence relate to
YOUR courses?"
13
SOLO (elaborated)
Note the list is non-exhaustive
QUANTITATIVE
QUALITATIVE
SOLO 2 uni-structural
SOLO 3 multi-structural
SOLO 4 relational
SOLO 5 extended abstract

• theorize
• generalize
• hypothesize
• predict
• judge
• reflect
• transfer theory (to new domain)

• analyze
• compare
• contrast
• integrate
• relate
• explain causes
• apply theory (to its domain)

• combine
• structure
• describe
• classify
• enumerate
• list
• do algorithm
• apply method

• define
• identify
• count
• name
• recite
• paraphrase
• follow (simple) instructions

14
SOLO verbs

• Mapped by
• B. Dahl C. Brabrand
• (Natural science context!)
• With help from
• 3 Educational research colleagues (medicine)
• J. Biggs C. Tang

15
SOLO Advantages

• Advantages of The SOLO Taxonomy
• Linear hierarchical structure (good for
  progression)
• Aimed at evaluating student learning
• Converges on research (at SOLO 5)

ResearchProduction ofnew knowledge
16
Exercise
T
"Key competences in YOUR course?"
Concurrency analyze systems compare models
17
The BLOOM Taxonomy (1956)

• The BLOOM Taxonomy

Analysis
Evaluation
Synthesis
SOLO 45
Qualitative
Application
Comprehension
Quantitative
SOLO 23
Knowledge

really intended to guide the selection of
items for a test rather than to evaluate the
quality of a students response to a particular
item -- (Biggs Collis, 1982)
18
Outline (Oct 10, 2013)

• 1) Introduction
• Constructive Alignment
• The SOLO Taxonomy
• 2) From Content to Competence
• Advocate a shift in perspective
• Elaborate on The SOLO Taxonomy
• 3) In Practice
• Concrete recommendations
• Alignment implementation process

19
Implementation Process
1) Think carefully about overall goal of
course (what students learn to do?)
2) Operationalize these goals and formulate
them as SOLO intended learning outcomes
alignment
learning incentive
learning support
3) Choose carefully the form(s) of assessment
( intended learning outcomes)
4) Choose carefully the form(s) of teaching
( intended learning outcomes)
20
Concrete Recommendations
1) Use 'standard formulation' a) puts
learning focus on the student b) competence
formulation "to be able to"

• Intended Learning Outcomes Genetics
  101
• After the course, the students are expected to be
  able to
• locate genes on chromosomes
• do simple calculations (e.g., recombination
  frequencies, in-breeding
  coefficients, Hardy-Weinberg,
  evolutionary equilibria).
• describe and perform connexion-analysis
• describe fundamental genetic concepts (e.g.,
  mutation variation,
  in-breeding, natural selection).
• describe and analyze simple inheritancies
• analyze inheritance of multiple genes
  simultaneously

4) Avoid 'understanding-goals' "To
understand X", "Be familiar with Y",
"Have a notion of Z" !
V
N
N
V
N
V
V
V
N
V
V
N
V
3) Use 'Verb Noun' formulation What the
student is expected to do with a given
matter .
2) List sub-goals as 'bullets' Clearer than
text
N
V
21
Implementation Process
1) Think carefully about overall goal of
course (what students learn to do?)
2) Operationalize these goals and formulate
them as SOLO intended learning outcomes
alignment
learning incentive
learning support
3) Choose carefully the form(s) of assessment
( intended learning outcomes)
4) Choose carefully the form(s) of teaching
( intended learning outcomes)
22
Teaching/Learning Activities
functional knowledge (problem oriented)
Project work
Case teaching
teacher centric
student centric
Exercise class
Lecture
declarative knowledge (discipline oriented)
23
Lecture (about Application)

• Teacher activity
• Introduce
• Explain
• Elaborate
• Discuss application
• Give examples
• Show PPT slides
• Questions on slides
• Winding up

• Student activity
• Listen
• Listen (maybe take notes)
• Understand? (correctly? deeply?)
• Listen (maybe take notes)
• Listen (maybe take notes)
• Watch (maybe note points)
• Write answers to questions
• Possibly ask a question

vs.
active teacher
vs.
Biggs Tang 2007, p.137
passive student
24
Learn about vs. Learn to do

• Learning (about)
• about application
• about cooking
• about programming
• about designing
• about analysis
• about construction
• about relating
• ...

• Learning (to do)
• to apply
• to cook
• to program
• to design
• to analyse
• to construct
• to relate
• ...

studentlistening
(to something about something)
vs.
student doing !!
25
Student Activation
"The (in-famous) Learning Pyramid"
Average retention rate
Note these percentages are "bogus"
5
Lecture
Listening Learning about
passive student
10
Reading
20
Audiovisual
student activation
30
Demonstration
active student
50
Discussion group
75
Practice by doing
Doing Learning to do
80
Teaching others
NTL Institute for Applied Behavioral Science,
Bethel, Maine
26
Constructive Alignment

• Make explicit ILO's ( )
• (and tell this to students)

Intended Learning Outcomes
ILO's
Teaching LearningActivities
Assessment
vs.
ROBERT extrinsically motivated
SUSAN intrinsically motivated
27
Questions...
Cognitive processes
Course descriptions
My research and teaching
"understanding" content ? competence
Association new old
The SOLO Taxonomy
'TLA' Teaching / Learning Activities
Teacher models levels 1 - 2 - 3
The Film
The Book
Susan Robert
?
Student activation
Tips'n'Tricks
CS v. NAT v. MAT
recite generalize
15 programming
Students at University
"What is good teaching?"
Constructive Alignment
John Biggs
Top Competences
28
Now, please "2-minute recap"
T

• Please spend 2' on thinking about and writing
  down the most important points from the talk
  now!

Immediately
After 1 day
After 1 week
After 2 weeks
After 3 weeks
29
Key References

• Teaching for Quality Learning at
  UniversityJohn Biggs Catherine TangSociety
  for Research into Higher Education, 2007.
  McGraw-Hill.
• Evaluating the Quality of Learning The SOLO
  TaxonomyJohn Biggs Kevin F. CollisLondon
  Academic Press, 1982
• Teaching Teaching Understanding
  UnderstandingClaus Brabrand Jacob Andersen19
  minute award-winning short-film (DVD)Aarhus
  University Press, Aarhus University, 2006
• "Constructive Alignment The SOLO Taxonomy a
  Comparative Study of University Competencies in
  Computer Science vs. Mathematics"Claus Brabrand
  Bettina DahlCRPIT, Vol. 88, ACS 3-17, R.
  Lister Simon, Eds., 2007

30
Thank You!
Film's homepage
((( http//www.daimi.au.dk/brabrand/short-film/
)))
31
Tips'n'Tricks (activation)

• Neighbour discussions

• Post-It exercise

• focus zoom in
• anonymous (!)
• swap'able
• everyone will engage
• empathetic control
• shared knowledge pool

• more questions (students dare ask them)
• better questions (students had a chance
  to discuss)

Phil Race
1-2 min timeout

• Frequent breaks

• Form variation

pulse reader measurements
lecturing blended with in-class activation
exercises
32
Tips'n'Tricks (cont'd)

• Use many examples(build on student
  pre-knowledge)

• Explicit structure

1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4.
wwwwwww
1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4.
wwwwwww
1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4.
wwwwwww
1. xxxxxxxxxx 2. yyyyyyyyyy 3. zzzzzzzzzz 4.
wwwwwww
?

• self evident to you teacher
• not to a learner student
• (esp. during learning process)

• "Less-is-more"

• Student 'recap' at end

• analyze
• compare
• relate

common deadlock, uncommon deadlock,
A-synchronization, B-synchronization, hand-shake,
multi-party synchronization, multi-party
hand-shake, binary semaphores, generalized
semaphores, blocking semaphores, recursive locks,
...
vs.
now
after 1 day
after 1 week
after 2 weeks
after 3 weeks
Emphasize depth over breadth (coverage)
33
Problematic Courses

• E.g. course Databases (at RUC/Roskilde)
• Note almost entirely non-operational(!)
• i.e. measure how?!

• obtain knowledge about the structure of
  database systems
• be familiar with design of databases by use of
  special notations like E/R and analysis
  through normalization
• get an overview of the most important database
  models and a detailed knowledge about the
  most important model - the relational model
  as well as the language SQL
• get an overview of database indexing and query
  processing
• obtain knowledge about application programming
  for DB systems.

Familiar with ?!
34
UNALIGNED COURSE
?
Teachers intention
Students activity

• e.g.
• explain
• relate
• prove
• apply

"Dealing with the test"
Exams assessment

• e.g.
• memorize
• describe

• e.g.
• memorize
• describe

35
ALIGNED COURSE
?
Teachers intention
Students activity

• e.g.
• explain
• relate
• prove
• apply

• e.g.
• explain
• relate
• prove
• apply

• e.g.
• explain
• relate
• prove
• apply

Exams assessment

• e.g.
• explain
• relate
• prove
• apply

• e.g.
• explain
• relate
• prove
• apply

36
Student Motivation

• Susan (intrinsic motivation)
• - wants to learn !
• Robert (extrinsic motivation)
• - to pass exams !

37
Constructivism

• Transmission is Dead
• (lectures )
• Knowledge is Actively Constructed !

!
active teacher passive students
risk
38
SOLO Taxonomy

• Hierarchy for Competences
• Deep learning (not surface) !

5 generalize, theorize, predict, 4 explain,
analyze, compare, 3 describe, combine,
classify, 2 recite, identify, calculate,
39
Stud Learning Focus

• Focus on Student Learning !
• (instead of what teacher does
• labelling students good/bad)
• Student activitation ? learning

40
Alignment

• Make explicit ILOs
• (Intended Learning Outcomes)
• (and tell this to students)

Exam ILOs Teaching
41
Acquisition of Competence

• Acquisition of competence progresses according to
  the following stages of learning
• 1) Unconscious incompetence
• 2) Conscious incompetence
• 3) Conscious competence
• 4) Unconscious competence
• 5) Capacity for moving consciously between
  stages 3) and 4) (which is required by a
  teacher)

42
On the Role of Examination

• Constructive Alignment
• A systemic theory (a teaching system w/
  cause/effects)
• A theory of planning (over the course of a
  course)
• A theory of motivation (and incentive)
• From the exam as a......to

"The exam does not come after, but before the
course!"
"Necessary evil"
application of alignment
Motivational and learning-guiding pedagogical
tool for the teacher(!)
43
Definition Good Teaching

• Definition
• Good news (we now know how to do this)
• Alignment!!!
• Explicitly defined course objectives (as verbs)!
• Discourage surface-learning!
• Encourage depth-learning!
• Less-is-more depth rather than breadth of
  coverage!

Good teaching is getting most students to use
the higher cognitive level processes that the
more academic students use spontaneously --
Teaching for Quality Learning at University,
John Biggs, 2003
44
The SOLO Taxonomy (1982)

• SOLO 1 (Pre-Structural)
• Here the subject does not have any kind of
  understanding but uses irrelevant information
  and/or misses the point altogether. Scattered
  pieces of information may have been acquired, but
  they are unorganized, unstructured, and
  essentially void of actual content or relation to
  a topic or problem.
• SOLO 2 (Uni-Structural)
• The subject can deal with one single aspect and
  make obvious connections. The subject can use
  terminology, recite (remember things), perform
  simple instructions/algorithms, paraphrase,
  identify, name, count, etc.
• SOLO 3 (Multi-Structural)
• At this level the subject can deal with several
  aspects but these are considered independently
  i.e., not in connection with one another.
  Metaphorically speaking the subject sees the
  many trees, but not the forest. He or she is able
  to enumerate, describe, classify, combine, apply
  methods, structure, execute procedures, etc.
• SOLO 4 (Relational)
• At level four, the subject may understand
  relations between several aspects and how they
  might fit together to form a coherent whole. The
  understanding forms a structure and now he or she
  sees how the many trees form a forest. A subject
  at this level may compare, relate, analyze, apply
  theory, explain in terms of cause and effect,
  etc.
• SOLO 5 (Extended Abstract)
• At this level, which is the highest, the subject
  may generalize structure beyond what was given,
  may perceive structure from many different
  perspectives, and transfer ideas to new areas. He
  or she may generalize, hypothesize, criticize,
  theorize, etc.