Incorporating IDEA into Assessment of General Education Learning Outcomes

1
Incorporating IDEA into Assessment of General
Education Learning Outcomes

• Stephanie Oetting, PhD
• Director of Institutional Research
• University of Saint Francis
• Fort Wayne, IN

2
General Aims

• Incorporating IDEA student ratings into general
  education assessment
• Using IDEA extra question process to collect data
  for general education assessment
• USF GE Assessment Model
• Mix of direct/indirect and internal/external
  measures
• Custom reports versus aggregate data files
• Using extra questions in many classes

3
Background

• General education curriculum in place 20 years
• Intense work for revision initiated in 2003
• New GE Framework approved late Spring 2007
• Assessment plan work successfully avoided
• Faculty Governance Structure
• Committee On Assessment Student Academic
  Achievement
• General Education Committee
• Academic Council

4
General Education Goals

1. Develop personal awareness of diverse/global
   society.
2. Demonstrate leadership, service, social
   responsibility.
3. Communicate effectively in personal/professional
   interactions.
4. Demonstrate competence in applying technologies.
5. Demonstrate personal and social awareness of
   importance of lifelong health/wellness.
6. Think analytically, synthetically, critically,
   and creatively in the pursuit of knowledge.
7. Develop awareness/understanding of
   artistic/expressive aspects of the human
   experience.
8. Appreciate spiritual dimension of life, be
   conscious of own religious perspective within a
   community context.

5
Layers of Complexity

• 8 General Education Goals
• operationalized with
• 31 Learning Outcomes
• grouped into
• 19 Distribution Requirements

6
State of Assessment

• Challenges
• Several significant administrative and structural
  changes
• New GE framework lacking assessment plan
• Exhaustion from GE framework revision
• No Clear leadership, accountability, authority
  for GE assessment and
• Committees searching for a responsible party!

• Strengths
• Program assessment well-developed in some
  programs
• Emerging group of faculty knowledgeable about,
  invested in assessment
• Support, encouragement from University
• AQIP Systems Portfolio in progress
• Assessment Opinion Leaders vocalizing

7
Getting Started

• You dont have to see the whole staircase, just
  take the first step.
• --Martin Luther King, Jr.

8
Task Force Convened

• Composition
• Goals
• BEGIN draft of assessment plan
• Reduce fear and avoidance
• Identify critical resources for implementation
• Cultivate culture of assessment
• Compensation reward

9
Requirements of Assessment Plan!

• Simple, seamless, sustainable
• Produce meaningful data, quickly and efficiently
• Use whats already in place, no extra time or
  work!
• Benefit students whenever possible
• Empower instructors/dept. to generate artifacts
• Need naturally occurring artifacts, not extra
  work
• Compensate faculty who give their time!
• No evaluation of faculty, course or program
• No policing or punitive approach

10
IF ONLY
11
Comforting thoughts

• It doesnt really matter whether you can
  quantify your results. What matters is that you
  rigorously assemble evidence quantitative or
  qualitative to track your progress.
• What matters is not finding the perfect
  indicator, but settling upon a consistent and
  intelligent method of assessing your output
  results, and then tracking your trajectory with
  rigor.
• Collins, J. (2005). Good to Great and the
  Social Sectors A Monograph(p. 7, 8)

12
GE Assessment Plan

• Direct measures
• Student demonstration of learning outcomes
• Evaluated by faculty
• Indirect measures
• Student perceptions
• Inferred measurements
• Internal measures
• Locally developed
• External measures
• Standardized assessments
• Results comparisons

13
Indirect Measures

• External
• Student self-reported progress on IDEA objective
  that aligns with goal
• Noel Levitz SSI
• Internal
• Student self-reported progress on learning
  outcome using IDEA extra question format
• Other surveys and assessments

14
Align Goals to IDEA Objectives

• Develop a personal awareness of our diverse and
  global society
• Developing a clearer understanding of, and
  commitment to, personal values (10)
• Demonstrate leadership, service, and social
  responsibility
• Acquiring skills in working with others as a
  member of a team (5)

15
Direct Measure

• Internal
• Artifacts generated at course level
• Assessed by faculty committee
• Holistic rubric using learning outcome
• Allows for comparison of faculty and student
  ratings

16
Goal 1
Develop a personal awareness of our diverse and
global society
LO1
LO3
IDEA Extra Q
IDEA Extra Q
Student satisfaction Noel Levitz SSI
62 Commitment to racial harmony on campus
Artifact iConnect Society, Natural Environment
Artifact History
MLK Survey
Artifact iConnect
IDEA Extra Q
Progress on IDEA objective 10 Understanding
of commitment to personal values
LO2
Direct Measure Indirect Measure
17
4-year Assessment Cycle

• 2007-2008 Goals 1 and 2
• 2008-2009 Goals 3 and 4
• 2009-2010 Goals 5 and 6
• 2010-2011 Goals 7 and 8
• 2011-2012 Goals 1 and 2
• . . .

18
Preliminary Results

• IDEA Objectives
• Percent selecting as important/essential
• Progress on relevant objectives
• Learning Outcomes
• IDEA Extra Questions student perception
• Artifact assessment faculty assessment

19
Results - Learning Outcomes
20
iConnect Course Summary Ratings

• Overall ratings for teacher and for course
• Student comments examined
• Faculty feedback sought

21

• Actionable Results
• Lessons Learned
• New Directions

22

• QUESTIONS?
• THOUGHTS?
• COMMENTS?