Assessing Evidence of Student Learning within The Context of Collective Educational Practices

1
Assessing Evidence of Student Learning within The
Context of Collective Educational Practices

• Presented at the April 28, 2006 Assessment
  Conference, CSU System
• Peggy Maki.
• PeggyMaki_at_aol.com

2
Foci

• Assessment of learning along the chronology of
  students studies (how students make meaning)
• Articulation of collaboratively agreed upon
  learning outcome statements
• Development of curricular-co-curricular maps that
  represent the fabric (coherence) that underlies
  students learning

3

• Development of methods that align with what and
  how students learn and receive feedback along the
  continuum of their learning (provide the basis
  for making different kinds of inferences)
• Development of criteria and standards of judgment
  that provide feedback to students about patterns
  of strengths and weaknesses, as well as provide
  feedback to educators about students actual
  achievement

4
Integrated Learning.
5
Learning in Disciplines

• Integration
• Synthesis
• Application
• Transfer
• Re-use of learning
• Re-positioning of understanding
• Construction of meaning

6
Educational Practices that Promote Integrated
Learning

• Pedagogy
• Curricular design
• Instructional design
• Educational tools
• Educational experiences
• Students Learning Histories/Styles

7
Development of Learning Outcome Statements
What Is a Learning Outcome Statement?

• Describes learning desired within a context
• Relies on active verbs (create, compose,
• calculate, construct, apply)
• Emerges from our collective intentions over time

8

• Can be mapped to curricular and co-curricular
  practices (multiple and varied opportunities to
  learn over time)
• Can be assessed quantitatively or qualitatively
  during students undergraduate and graduate
  careers
• Is written for a course, program, or institution

9
Levels of Learning Outcome Statements
10
Sample APA Learning Outcome Statements

• 1.3 Use the concepts, language, and major
  theories of the discipline to account for
  psychological phenomena.
• 1.4 Explain major perspectives of psychology
  (e.g., behavioral, biological, cognitive,
  evolutionary, humanistic, psychodynamic, and
  sociocultural).
• http//www.apa.org/ed/critique_study.html

11
Sample Biology Learning Outcome Statements

• Diagram and explain the major cellular processes
  in eukaryotes and prokaryotes.
• Apply the scientific process, including designing
  and conducting experiments and testing
  hypotheses.

12
Ways to Articulate Outcomes

• Adapt from professional organizations
• Derive from mission of institution/program/departm
  ent/service
• Derive from students work
• Derive from ethnographic process
• Derive from exercise focused on listing one or
  two outcomes you attend to
• Draw from taxonomies, such as Blooms

13
In one sentence articulate one disciplinary
learning outcome statement that you expect your
students to be able to demonstrate by the end of
their program of study with you
14
Development of Maps and Inventories of
Educational Practices

• Reveal how we translate outcomes into educational
  practices offering students multiple and diverse
  opportunities to learn
• Help us to identify appropriate times to assess
  those outcomes
• Identify gaps in learning or opportunities to
  practice

15

• Help students understand our expectations of them
• Place ownership of learning on students
• Enable them to develop their own maps or learning
  chronologies

16
Identification of Assessment Methods

• Every assessment is also based on a set of
  beliefs about the kinds of tasks or situations
  that will prompt students to say, do, or create
  something that demonstrates important knowledge
  and skills. The tasks to which students are asked
  to respond on an assessment are not arbitrary.
• National Research Council. Knowing what
  students know The science and design of
  educational assessment . Washington, D.C.
  National Academy Press, 2001, p. 47.

17
Assumptions Underlying Teaching
Actual Practices
Assumptions Underlying Assessment Tasks
Actual Tasks
18
What Tasks Elicit Learning You Desire?

• Tasks that require students to select among
  possible answers (multiple choice test)?
• Tasks that require students to construct answers
  (students problem-solving and thinking
  abilities)?

19
Approaches to Learning

• Surface Learning
• Deep Learning

20
When Will or Do You Seek Evidence?

• Formativealong the way?
• For example, to ascertain progress
• or development
• Summativeat the end?
• For example, to ascertain mastery level of
  achievement

21
Some Options

• E-Portfolios
• Capstone projects (mid-point and end-point) or
  culminating projects (solo or team-based)
• Performances, productions, creations
• Visual representations (mind mapping, charting,
  graphing)

22

• Disciplinary or professional practices, such as
  laboratory reports or field reports
• Agreed upon embedded assignments such as in
  service learning experiences (see handout)
• Writing to speaking to visual presentation

23

• Team-based or collaborative projects
• Internships and service projects
• Critical incidents
• Chronological responses to a problem or issue
• Written responses to a prompt

24

• Externally or internally juried review of student
  projects
• Externally reviewed internship
• Performances on a case study/problem
• Performances on a case study accompanied with
  students analysis

25

• Performance on national licensure examinations
• Locally developed tests
• Standardized tests
• Pre-and post-tests
• Learning Logs or Journals

26

• Oral defense/response (sometimes as part of a
  capstone project)
• Self-reflective writing (often accompanies
  student work or occurs after feedback)
• Colloquia (chemistry example)
• De-construction of a problem or issue

27

• Problem with solution and ask for other solutions
• Inferences from a discourse selection
• Data mining (learning object sites)

28
Identify Methods to Assess Your Outcomes

• Referring to pages 20-27, identify both direct
  and indirect methods you might use to assess one
  or more of the outcomes you developed today
• Determine the kinds of inferences you will be
  able to make based on each method.

29
Development of Standards and Criteria
of Judgment

• Scoring rubrics--A set of criteria that
• identifies the
• (1) expected characteristics/traits of student
  work/behavior
• (2) levels of achievement along those
• characteristics/traits

30

• Provide a means to assess the multiple dimensions
  of student learning.
• Are collaboratively designed based on how and
  what students learn (based on curricular-co-curric
  ular coherence)
• Are aligned with ways in which students have
  received feedback (students learning histories)

31

• Are useful to students, assisting them to improve
  their work and to understand how their work meets
  performance expectations (can provide a running
  record of achievement).
• Raters use them to derive patterns of student
  achievement to identify strengths and weaknesses
  and thus verify the efficacy of educational
  practices as well as those that need to be
  changed

32
Scoring Sheet (rubric)
33
Development of Scoring Rubrics

• Emerging work in professional and disciplinary
  organizations
• Research on learning (from novice to expert)
• Student work
• Interviews with students
• Experience observing students development

34
The Process
Gather Evidence
Interpret Evidence
Mission/Purposes Learning Outcomes
How well do students achieve our outcomes?
Enhance teaching/ learning inform institutional
decision- making, planning, budgeting
35
What and how students learn depends to a major
extent on how they think they will be assessed.
John Biggs, Teaching for Quality Learning at
University What The Student Does. Society for
Research into Higher Education Open University
Press, 1999, p. 141.