Assessing for Learning

1
Assessing for Learning

• Presented by Peggy L. Maki June 7, 2004
• Seattle Pacific University
• PeggyMaki_at_aol.com

Material from Maki, P. (2004). Assessing for
Learning Building a Sustainable Commitment
Across the Institution. Stylus Publishing and
AAHE.
2
How Do You Learn?

• _______________________________
• ________________________________
• ________________________________
• ________________________________

3
Topics

• Research on Learning That Informs the
  Relationship among Teaching, Learning, and
  Assessment
• Curricular and Co-curricular Coherence (Maps and
  Inventories)

4

• Development of Learning Outcome Statements
• Alignment of Assessment Methods
• with Learning Outcome Statements and
  Collective Educational Practices

5
Terms

• Learning Outcome Statementssentences that
  describe what we expect our students to
  demonstrate, represent or produce as a result of
  how and what they learn.
• Coherencedegree to which the curriculum and
  co-curriculum intentionally offer multiple and
  varied opportunities for students to learn what
  the institution and its programs and services
  assert they teach

6

• Maps and Inventoriesstrategies that enable us to
  ascertain how well we intentionally develop what
  we assert we expect students to learn
• Alignmentdegree to which learning outcome
  statements match how and what we teach degree to
  which assessment methods methods match our
  collective educational practices

7
Research on Learning That Informs the
Relationship among Teaching, Learning, and
Assessment

• Learning is a complex process of
  interpretation--not a linear process
• Learners create meaning as opposed to receive
  meaning
• 1.

8

• Knowledge is socially constructed (importance of
  peer-to-peer interaction)
• People learn differentlyprefer certain ways of
  learning (learning inventories, such as Kolb or
  Vark)

9

• Deep learning occurs over timetransference
• Meta-cognitive processes are a significant means
  of reinforcing learning (thinking about ones
  thinking)

10

• Learning involves creating relationships between
  short-term and long-term memory
• Transfer of new knowledge into different
  contexts is important to deepen understanding

11

• Practice in various contexts creates expertise
• Surface Learning
• Deep Learning

12
Curricular and Co-Curricular Coherence

• What do you do in your classes or in your
  programs or services to promote the kinds of
  learning or development that the institution
  seeks?
• What do you expect your students to know and be
  able to do by the end of their education at your
  institution?

13

• What do the curricula and other educational
  experiences add up to?
• Which students benefit from specific teaching
  strategies or educational experiences?
• What educational processes are responsible for
  the intended student outcomes the institution
  seeks?

14

• How can you help students make connections
  between classroom learning and experiences
  outside of the classroom?
• What pedagogies/educational experiences develop
  knowledge, abilities, habits of mind, ways of
  knowing/problem solving?

15

• How are curricula, pedagogy, and educational
  experiences designed to develop knowledge,
  abilities, habits of mind, ways of knowing?
• How do you intentionally build upon what each of
  you teaches or fosters to achieve programmatic
  and institutional objectives?

16

• What methods of assessment capture desired
  student learning--methods that align with
  pedagogy, content, and curricular design?

17
When a Student Becomes a Biologist, Psychologist,
Engineer..
18
Maps and Inventories

• Help us determine coherence among our educational
  practices that enable us to design appropriate
  assessment methods
• Provide a visual representation of students
  journey

19

• Help students make meaning of the journey
• Help students develop their own learning map

20
Developing Learning Outcome Statements
21
What Is an Outcome Statement?

• A sentence that describes what students should
  demonstrate, represent, or produce based on how
  and what they learn.
• Emerges from what we value and how we teach or
  students learn that is, it emerges from our
  educational practices and is developed through
  consensus.

22

• Relies on active verbs, such as create, compose,
  calculate, develop, build, evaluate, translate,
  etc., that target what we expect students to be
  able to demonstrate
• Is written for a course, program, or institution

23

• Is aligned with curriculum and co-curriculum
• --varied and multiple ways to
• learn that desired outcome
• Is based on intentionality (established through
  dialogue)

24

• Helps students understand expectations and holds
  them accountable for their learning
• Can be assessed quantitatively or qualitatively

25
Example from ACRL

• ONE OUTCOME Student examines and compares
  information from various sources in order to
  evaluate reliability, validity,accuracy,
  timeliness, and point of view or bias.

26
Write one outcome statement that captures what a
graduate from your institution or a program
should be able to demonstrate, represent, or
produce
27
How well does your outcome statement meet
characteristics of a good statement?

• Describes learning desired within a context
• Relies on active verbs

28

• Emerges from your collective intentions
• Can be mapped to curricular and co-curricular
  practices (multiple and varied opportunities to
  learn over time)
• Can be assessed quantitatively or qualitatively
  during students undergraduate and graduate
  careers

29

• The tasks to which students are asked to
  respond on an assessment are not arbitrary. They
  must be carefully designed to provide evidence
  that is linked to the cognitive model of learning
  and to support the kinds of inferences and
  decisions that will be based on the assessment
  results.
• National Research Council. Knowing what
  students know The science and design of
  educational assessment . Washington, D.C.
  National Academy Press, 2001, p. 47.

30
Assumptions Underlying Teaching
Actual Practices
Assumptions Underlying Assessment Tasks
Actual Tasks
31
Development of Assessment Methods

• Align methods with learning outcome statements
  and students educational experiences (verified
  through maps or inventories)

32
When Will You Seek Evidence?

• Formativealong the way?
• For example, to ascertain
• progress or development
• Summativeat the end?
• For example, to ascertain
• mastery level of achievement

33
What Tasks Elicit Learning You Desire?

• Tasks that require students to select among
  possible answers (multiple choice test)?
• Tasks that require students to construct answers
  (students problem-solving and thinking
  abilities)?

34
Some Methods That Provide Direct Evidence

• Student work samples
• Collections of student work (e.g. Portfolios)
• Capstone projects

35

• Course-embedded assessment (derive examples
  develop agreed upon prompt and ask students to
  respond to it in class or at a designated time)
• Observations of student behavior
• Internal juried review of student projects

36

• External juried review of student projects
• Externally reviewed internship
• Performance on a case study/problem
• Performance on case study accompanied with
  students analysis

37

• Performance on national licensure examinations
• Locally developed tests
• Standardized tests
• Pre-and post-tests

38

• Response to critical incident
• Essay tests blind scored across units
• Mapping
• Learning Logs or Journals
• Writing, Speaking, Visual Representation

39
Methods That Do Not Provide Direct Evidence but
May Be Combined with Other Methods-Indirect
Methods

• Faculty publications (unless students are
  involved)
• Courses selected or elected by students
• Faculty/Student ratios

40

• Percentage of students who study abroad
• Enrollment trends
• Percentage of students who graduate within
  five-six years
• Diversity of student body

41

• Focus group (representative of the population)
• Interviews (representative of the population)
• Surveys

42

• Other sources of information that contribute to
  your inference making NSSE results, grades,
  participation rates or persistence in support
  services, course-taking patterns, majors

43
What and how students learn depends to a major
extent on how they think they will be assessed.
John Biggs, Teaching for Quality Learning at
University What The Student Does. Society for
Research into Higher Education Open University
Press, 1999, p. 141.
44
Works Cited

• Biggs, J. (1999). Teaching for Quality
  Learning at University What The Student Does.
  Society for Research into Higher Education Open
  University Press, 1999, p. 141.
• Maki, P. (forthcoming, 2004., May). Assessing
  for Learning Building a Sustainable Commitment
  Across the Institution. Sterling, VA Stylus
  Publishing, LLC, and the American Association for
  Higher Education.
• National Research Council. 2001. Knowing What
  Students Know The Science and Design of
  Educational Assessment. Washington, D.C.
  National Academy Press