AAC

1
(No Transcript)
2
AACU 2006 General Education and Outcomes That
Matter in a Changing World
Using Student Learning to Assess General
Education Courses Cynthia Conn, Associate
Director Katie Desmond, Graduate Research
Assistant Office of Academic Assessment Northern
Arizona University
3
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• Advance Organizer
• This session focuses on assessing student
  learning in general education courses (rather
  than the overall assessment of a general
  education program).
• Key questions
• What techniques are sustainable?
• How can assessment at the courses level feed into
  general education and degree level program
  assessment?
• Are specific classroom practices --- pedagogical
  and assessment techniques --- that align with the
  overall mission of the program being encouraged?
• After participating in this session you will be
  able to
• Suggest approaches for getting started, revising
  or continuing course level assessment efforts
• Describe various course level assessment
  techniques and sampling procedures
• Consider how to align course level assessment
  with program level assessment efforts

4
AACU 2006 General Education and Outcomes That
Matter in a Changing World
Ice Breaker Create a name card, then At your
table, introduce yourself, the institution you
are affiliated with, and the one question you
have regarding assessment of general education
courses.
5
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• Participant Questions
• How to aggregate data from a course and what to
  do with it?
• How do we aggregate data across disciplines,
  courses and assignments?
• How do we conduct assessment of multiple
  sections of the same course?

6
AACU 2006 General Education and Outcomes That
Matter in a Changing World
Where do you start? Are the student learning
outcomes for the skills, themes, distribution
blocks, content areas, etc. (i.e., whatever you
want to assess related to your general education
curriculum) defined?
7
AACU 2006 General Education and Outcomes That
Matter in a Changing World
How/who developed the student learning outcome
definitions? Are there any resources/
references that you would recommend for campuses
getting started?
8
AACU 2006 General Education and Outcomes That
Matter in a Changing World
Are your general education student learning
outcome definitions publicly available via your
institutions website?
9
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• What is an Assessment Audit?
• What are you already doing at your institutions
  for assessment?
• For example, look at the embedded assignments
  (those that already exist) rather than adding
  on an assignment.
• What is going on at your institution so you can
  identify the gaps in assessment. It may tell you
  that you need more direct assessment.

10
AACU 2006 General Education and Outcomes That
Matter in a Changing World
What is an Assessment Audit?
Investigation of the assessment efforts
currently being implemented or assessment data
currently being gathered intended as a
springboard for determining useful processes as
well as gaps that should be addressed (Walvoord,
2004).
11
AACU 2006 General Education and Outcomes That
Matter in a Changing World
Has your institution conducted an Assessment
Audit? Think-Pair-Share (Clock Buddies)
Discuss ideas related to what you have done, are
planning to do, or could do in terms of
conducting an Assessment Audit. Discuss
specifically how you investigated (or could
investigate) what is being done in terms of
course level assessment.
12
Assessment audits what is going on?

• Surveying faculty or dept. chairs and getting
  buy in. During this inventory, making sure you
  are getting valuable feedback. (What is
  happening in those courses?)
• Range within multiple section courses ? through a
  faculty retreat, creation of a standardized
  rubric applied to a synthesized assignment
  (norming that can bring credibility to data).
• A question on an exam that revealed something
  about students perception of ethics led to a
  larger conversation about how this skill was
  being taught in the discipline.
• Investigate syllabi asking how they address the
  elements. Is this course meeting our
  requirements and contributing to our assessment
  needs?
• Do our assessment mechanisms measure our
  students best work a valid representation of
  what they can do?
• Using the same rubric that is used to assess the
  SLO have the student respond to the rubric
  did you learn this skill (to write effectively,
  etc) as an indirect source for assessment data.
• Review portfolios and web-pages (any major
  project in that course)

13
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• Assessment Audit Ideas

14
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• NAU Assessment Audit
• Content analysis study of course syllabi
• Report summarizing what instruments are currently
  in use, how data is being used, and how data
  could be compared to data from other instruments

15
AACU 2006 General Education and Outcomes That
Matter in a Changing World
Coherent Curriculum Think-Pair-Share (Clock
Buddies) What (useful) policies and procedures
are in place at your institution that establish
expectations for course level assessment?
16
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• Policies Procedures Ideas
• Program review course outlines have to be
  revised. Assessment is tied into this. (5 year)
  
• Every dept. has a member of the assessment
  committee. (Gen Ed committee is different)
• Appoint an assessment leader on campus.
• Pressure from above!
• Expectations work best coming from faculty.
• Content analysis of syllabi.
• Start with the programs that are already doing
  this nursing is a field that has good assessment
  procedures.

17
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• Content Analysis Study Findings
• Create explicit expectations related to
  participating in assessment
• Institute procedures for the re-review of syllabi
  (e.g., every 3 or 5 years)
• Develop policies regarding the use of appropriate
  pedagogical methods --- are the selected skill
  and content areas explicitly taught or just
  rehearsed? (e.g., Is feedback on a paper enough
  to constitute teaching effective writing?)
• Require the use of the syllabus template, and
  demonstration of alignment between course
  objectives and general education student learning
  outcomes, listing of skill as well as content,
  and appropriate assessment(s).
• Create a database of current general education
  course syllabi that is publicly accessible by
  faculty members and students
• Determine entity responsible for implementing the
  assessment plan and reporting results

18
AACU 2006 General Education and Outcomes That
Matter in a Changing World
Whats next? What did your Assessment Audit tell
you specifically related to course level
assessment? Are your research/assessment
questions related to course level assessment
documented? (see Ross Leskes (2005), Levels of
Assessment From Student to the Institution, pp.
7-8) How will course level assessment data be
gathered and used for program level assessment?
19
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• Round Robin (at your table) What sustainable
  course level assessment techniques are being
  used, have been proposed, or could be employed at
  your institution? Discuss both the specific
  techniques as well as the logistics for
  implementing the assessment strategy.

20
AACU 2006 General Education and Outcomes That
Matter in a Changing World
Course Assessment Ideas
21
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• Techniques Being Considered or Employed at NAU
• Collecting data related to embedded assignments
  via the use of rubrics
• Creating survey questions that are more closely
  aligned with new definitions possibly request
  over sampling of specific target population for
  Sophomore Survey
• Add questions regarding general education
  student learning outcomes to course evaluations

22
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• Round Robin (at your table) What types of
  sampling techniques are you using or what would
  be appropriate sampling techniques?

23
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• Sampling Techniques Ideas

24
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• NAUs Sampling Approach for Implementation of
  Rubrics
• 20 of students, per skill, in fall and spring
  semesters, representative of delivery modes
  (Flagstaff Mountain, web delivered, Yuma, and
  statewide face to face or IITV), random start,
  cycle through all courses and repeat

25
AACU 2006 General Education and Outcomes That
Matter in a Changing World
How do you close the assessment loop? Large
group discussion How is (or could) course level
assessment data be gathered and used for general
education program level assessment? degree level
program assessment?
26
AACU 2006 General Education and Outcomes That
Matter in a Changing World

• Using Assessment Data Ideas