A Model for Program Outcomes Assessment

1
A Model for Program Outcomes Assessment

• Pam Bowers, Ph.D.
• Director, University Assessment Testing
• Oklahoma State University
• www.okstate.edu/assess

2
Identify Outcomes
Improve Program
Relate To Program
Assess
Where Assess?
Identify or develop Tool (exam, rubric)
How Assess?
Who Evaluates?
3
Identify Outcomes
4
Identify Outcomes
Relate To Program
5
Student Learning Outcomes

• Describe knowledge, skills, attitude
• this is what is assessed in program outcomes
  assessment
• Can be mapped to curriculum
• Measureable
• students must be able to demonstrate learning in
  a way that can be assessed

6
Example Graduates will be able to communicate
effectively in writing

• Describe knowledge, skills, attitude
• students will be able to
• Can be mapped to curriculum
• writing assignments throughout program provide
  opportunities for increasing knowledge/improving
  skills

7
Identify Outcomes
Relate To Program
Where Assess?
8
When / How to Assess

• Can be mapped to curriculum
• What courses/experiences contribute most to this
  knowledge/skill?
• Measureable students demonstrate learning in
  ways that can be assessed
• What assignments/activities demonstrate student
  achievement of this outcome?
• (papers, projects, exams, portfolio artifacts)

9
Identify Outcomes
Relate To Program
Where Assess?
How Assess?
10
When / How to Assess

• What context provides best opportunity for
  measurement?
• capstone course, internship, certification
  process
• What assignments or program experiences best
  demonstrate student achievement of the learning
  outcome?
• capstone course project, licensure exam,
  internship evaluation

11
Identify Outcomes
Relate To Program
Where Assess?
How Assess?
Who Evaluates?
12
Who conducts the assessment?

• Who provides valid, reliable measure of student
  achievement of the learning outcome being
  assessed?
• program faculty
• testing center
• external reviewer
• internship supervisor
• Consider ease of administration, cost
  effectiveness, credibility

13
Identify Outcomes
Relate To Program
Where Assess?
Identify or develop Tool (exam, rubric)
How Assess?
Who Evaluates?
14
Identify or develop assessment tool

• What characteristics/criteria define
  achievement of the learning outcome?
• What work/activity (assignment(s), exam(s),
  internship) requires students to demonstrate
  expected skills/knowledge/attitudes?
• How can work be evaluated to provide useful
  information for program improvement?
• What data is provided by this evaluation?
• areas of strength/weakness, overall achievement

15
Identify Outcomes
Relate To Program
Assess
Where Assess?
Identify or develop Tool (exam, rubric)
How Assess?
Who Evaluates?
16
Conducting the Assessment

• Program outcomes assessment
• timing when in the program will students have
  maximum achievement of this outcome?
• Representative sample
• results will be most useful for program
  improvement if representative of all grads
• Systematic data collection process
• reliable, valid evaluation method
• qualified/trained evaluators
• provides data indicating areas of strengths and
  weaknesses in student learning (all students)

17

• An approximate answer to the right question is
  worth a good deal more than an exact answer to an
  approximate question.'
• --a paraphrase of J. W. Tukey (1962)

18
Example Method Measure studentsability to
communicate effectively in writing by evaluating
students work using a faculty developed rubric.

• Work to be evaluated selected writing
  assignment in capstone course, or
  theses/dissertations (toward program end)
• Sample all seniors or random sample
• Tool faculty designed rubric - evaluation
  criteria are defined by faculty (validity)
• Evaluators trained faculty or external reviewers
  (reliability)
• Data provided scores on criteria indicating
  students learning strengths/weaknesses

19
Identify Outcomes
Improve Program
Relate To Program
Assess
Where Assess?
Identify or develop Tool (exam, rubric)
How Assess?
Who Evaluates?
20
Use Results to Improve

• Do results indicate achievement of learning
  outcomes as expected by faculty?
• How will faculty use results for program
  improvement?
• Changes to focus on strengths?
• Changes to improve learning in weaker areas?
• Improve/change assessment methods?
• Modify Expected Program Learning Outcomes?

21
Identify Outcomes
Improve Program
Relate To Program
Assess
Where Assess?
Identify or develop Tool (exam, rubric)
How Assess?
Who Evaluates?
22
Identify Outcomes
Improve Program
Relate To Program
Assess
Where Assess?
Identify or develop Tool (exam, rubric)
How Assess?
Who Evaluates?
23

• In God we trust
• all others bring data.
• Margaret Spellings
• U.S. Secretary of Education