Faculty InService in lieu of regular Committee Meeting

1
Compliance Certification Committee
Faculty In-Service in lieu of regular Committee
Meeting January 10, 2007 900-430 Beto
Academic Center
2
Agenda
Faculty In-service January 2007 Compliance
Certification Committee

• 900-915 Opening Prayer Fac Business Joel
  Heck
• 915-945 Opening Comments Tom Cedel
• 945-1020 Accreditation Update
• -- 20 min B R E A K --
• 1040-1200 Student Achievement
• Program Alignment Institutional Backmap
• Group Exercise completing the backmap
• -- 1200-100 L U N C H --
• 100-210 Measuring Outcomes
• Aggregating individual student assessment
• Group Exercise Writing Rubrics
• -- 20 min B R E A K --
• 230-330 Core Assessment
• CAAP Overview
• Group Exercise Completing the CAAP Planning
  Workbook

3
Opening Comments
Faculty In-service January
2007

• Be informed
• About your area
• About the university
• Be involved
• Assessment of student learning
• Program assessment
• Concordias mission
• Be prompt
• The time is now

4
Mission Framework

• Teach
• Faculty approved learning outcomes
• Map of outcomes and courses
• Model
• Faculty and staff development
• Practice
• Integration of curricular and co-curricular
• Recognize
• Award programs, speaker series

5
Accreditation Progress Report
Faculty In-service January
2007

• 58 of the 74 Principles of Accreditation have a
  visible path toward in compliance
• 16 areas fall short of compliance
• 3 Common threads outcomes measurement
  equivalency for all consistency
• Argumentative Outlines in development
• First draft of narratives due by March
• Responsibility Matrix
• Marries corporate owners, program directors, and
  compliance committee

6
Faculty In-service January
2007
General Education-Donna Janes

• Comprehensive Standard 3.5.1 Competencies within
  General Education Core

Institutional Effectiveness-Mike Moyer Gayle
Grotjan

• Core Requirement 2.5 Institutional Effectiveness
  
• Comprehensive Standard 3.3.1 Outcomes Assessment
  Analyses for Improvement
• Comprehensive Standard 3.4.1 Approved by
  Faculty Learning Outcomes
• Federal Standard 4.1 Evaluation of Student
  Achievement

7
Faculty In-service January
2007
Degree Programs-Yusheng Feng

• Core Requirement 2.7.2 Program Content

Faculty Credentials-Larry Meissner

• Core Requirement 2.8 Faculty
• Comprehensive Standard 3.7.1 Qualifications

Library Learning Resources-Curt Giese

• Core Requirement 2.9 Learning Resources and
  Services
• Comprehensive Standard 3.4.14 Use of Technology
  to Enhance Student Learning
• Comprehensive Standard 3.8.2 Access to
  Instruction in Use

8
Faculty In-service January
2007
Contractual Arrangements-Linda Lowery

• Comprehensive Standard 3.4.7 Consortial and
  Contractual Educational Programs

Policies- Eric Silber

• Comprehensive Standard 3.4.3 Admissions Policies
• Comprehensive Standard 3.4.10 Defines Publishes
  General Major Program Requirements

Administration Employees-Shirley Carey

• Comprehensive Standard 3.2.10 Institution,
  Evaluation of Administrators
• Comprehensive Standard 3.2.14 Institution,
  Ownership of Materials

9
Accreditation Timeline

• Kick-off completed in Jan 2006
• Compliance Audit completed in Sep 2006
• QEP exploration completed in Nov 2006
• QEP topic selection Jan 2007
• First draft of 74 narratives in Mar 2007
• Compliance Certification due to SACS in Sep 2007
• Off-site review in Nov 2007
• Focus report in Dec 2007
• QEP due to SACS in Dec 2007
• On-site review during the window of Jan-Apr 2008
• SACS reaffirmation results published in Dec 2008

10
Responsibility Marriage Matrix

• For each Principle (74) identify
• Corporate owners
• Supporting corporate office
• Primary and supporting program
  directors/managers
• Compliance Committee facilitator
• Owners shape the tone and content of the
  narrative
• Directors/managers assemble the supporting
  arguments and evidentiary proof
• Topical committee members are facilitators for
  the narrative process, and a devils advocate for
  the owner
• The ubiquitous all have a role in the process

11
Challenges

• Challenges to find the path toward compliance
• Linkage of programs (academic support) to
  institutional goals and institutional learning
  outcomes
• Measurement of outcomes a cyclical record of
  using results to shape a culture of quality
  enhancement
• Faculty Credentials
• Ownership Accountability
• Time management
• Closing the Looptwo cycles

12
Wild Cards

• Substantive Change
• Remote Sites
• M Ed and Nursing Program
• Relocation
• Distance Learning
• Demonstrating equivalency for all students in
  academic support programs
• New SACS institutional rep changes to the
  Principles of Accreditation
• Changes in the corporate structure

13
QEP Update

• Comments from the QEP Committee co-chairs
• Don Christian Cathy Brigham

14

• BREAK
• 20 minutes

15
Student Achievement

• college is meaningless without a curriculum, but
  is more so when it has one that is meaningless
• a curriculum creates a world. It is important
  then that it has a center and an order or parts.
  Some studies are surely secondary to others, as
  some rest on others as a base. This should be
  made manifest, and no student should be permitted
  to ignore the primary, the basic matter.
• Van Doren, 1943 (as quoted in Birnbaum, 2004, p.
  118)

16
Student Achievement

• As we pursue the path of growth from program
  effectiveness to institutional effectiveness, we
  have the opportunity to change the question from
  What students know and can do to What students
  know and can do as a result of their educational
  experiences at Concordia
• Barnes, et al. (2006), p. 6
• The first question is one linked to individual
  student achievement course outcomes
• The second question is one linked to the
  collective body of students institutional,
  program outcomes

17
Curriculum Alignment
What we say students should learndesired
institutional learning outcomes
What is actually taughtcourse objectives
DECLARED Curriculum
TAUGHT Curriculm
LEARNED Curriculum
What students actually learnmeasured core and
program outcomes
The overlap is where consistency and
intentionality happenalignment improves the
overlap
Adapted from Barnes, et al. (2006), p. 9
18
Curriculum Alignment

• Questions
• Do the core and programs of study provide an
  intentional and consistent path for students to
  learn what we assert we inculcate?
• Does the curriculum provide the means to build on
  previous learning and to reinforce outcomes?
• Do programs and associated courses promote
  institutional-level knowledge, skills, and
  attitudes that shape habits of the mind and
  behavior?
• How well do institutional expectations translate
  into intentional educational practices at the
  program and course level?

Adapted from Barnes, et al. (2006), p. 8
19
Curriculum Alignment

• Outcomes
• Accountabilitynot about how you do your job, but
  about consistency between multi-section courses
  and between Full and Part-time faculties
• Congruence between Institutional Outcomes and
  Program/Course Goals
• Curriculum awareness that clarifies the
  relationship between teaching learning,
  identifies gaps, ensures sequencing, and promotes
  academic rigor
• Curriculum effectiveness that promotes upward
  movement in the cognitive, affective, and motor
  taxonomies
• Curriculum coherence that promotes the
  institutional mission and the academic excellence
  goals and objectives

Adapted from Barnes, et al. (2006), p. 10-14 and
from SACS-COC (2005), Relevant Questions for CR
2.5 2.7
20
Back-mapping

• Execution
• You, as a faculty, have already done a lot of
  this
• Core curriculum outcomes that were developed in
  2002/3 are the basis upon which we move forward
• The strategic plan and its associated academic
  excellence vector have your 9 outcomes plus 1 at
  the center
• The first round of back-mapping was eye-opening
• Today, well do round 2
• Gen Ed, IE, Degree Program requirements
• Comprehensive Standard 3.5.1 Competencies within
  General Education Core
• Comprehensive Standard 3.4.1 Approved by
  Faculty Learning Outcomes
• Core Requirement 2.7.2 Program Content

21
Back-map Exercise

• Form sub-groups by colleges divisions
• Primary focus is on the columns for b.i through
  b.ix
• Place an X in the cell of the 1-3 substantive
  areas that your program/major supportswhere you
  place an X indicates areas that you will employ
  some means of aggregated data collection to
  measure learning outcomes.
• Majors should have an X in b.x.
• BA/BS/AA Core has its own section on page 3

22

• LUNCH
• Resume at 1 PM

23
Measuring Outcomes

• Moving toward Institutional Effectiveness by
  using aggregated student/course assessment data
• WHY?
• We already do a lot of individual assessment of
  students, instructors, courses
• We can use what you are already familiar with
• Save ourselves a lot of work
• Earn double mileage

Adapted from Poe George (2006), p. 3-4
24
Measuring Outcomes

• Student
• Students assessed with respect to their
  individual mastery of material presented ensure
  capability or competency of each student
• Accomplished through exams, projects, essays,
  presentations, etc.
• Results are used to assign grades, remedial
  action, intervention, etc.
• INDIVIDUAL growth and improvement

• Program
• Institutional performance in meeting its goals
• Accomplished through exams, projects, essays,
  presentationsBUT with the data aggregated
• Results are used for program improvement, and as
  a basis for enhancing awareness, alignment, and
  coherence of the curriculum
• PROGRAM improvement

Adapted from Poe George (2006), p. 5-6
25
Measuring Outcomes

• What is an outcome?
• Student Learning Outcomes are NOT
• Individual course outcomes or course learning
  objectives
• Individual student assessment within the context
  of a given course
• Student Learning Outcomes ARE
• Over-arching end results of the academic program,
  i.e. the 9 1 outcomes
• Faculty/Administration/Regents view of the most
  important attributes of the students as a
  finished product of Concordia

Adapted from Poe George (2006), p. 15
26
Measuring Outcomes

• The student learning outcomes of programs are
• What students should
• Know (Cognitive)
• Think Feel (Affective)
• Be able to do (Motor)
• You measure these routinely for individual
  students relative to your courses and within your
  disciplines/programs
• KEY ISSUE To the extent that a given student
  assessment measures or relates to an
  institutional student learning outcome, it can
  potentially be aggregated and used for program
  assessment

Adapted from Poe George (2006), p. 20-21
27
Measuring Outcomes

• Commonly used student assessments that might
  be useful for program assessment
• Comprehensive Exams
• Internships
• Essays
• Portfolios
• Oral presentations Juried Performances
• Senior or Capstone Courses
• Embedded assessment tools
• Theses

Adapted from Poe George (2006), p. 23
28
Measuring Outcomes

• There is no free lunch, however
• Using student assessments as a deliberate means
  of program evaluation requires linking of a
  portion of that assessment to institutional
  outcomes and specifying the criteria for success
• the delta between what you expected and what
  you measured becomes the basis for assessing the
  program

29
Measuring Outcomes

• Key issues
• Means of assessment must relate to or measure
  broader outcomes
• Criteria for success must be stated in terms of
  averages or aggregates of individual student
  scores
• Scoring standards should be based on faculty
  consensus
• Scoring validity is enhanced with more than one
  faculty member involved in the scoring process
• Outcome scoring can be transparent to the student
  and independent of the individuals course grade

30
Measuring Outcomes

• Guidelines
• Select a representative sample across a program
  (some, most, all) and score relative to the
  criteria for success
• Internal or External evaluators add value
• Its not about you personallyits about a
  program and the contribution each of the
  programs parts makes toward the fulfillment of
  institutional goals.

31
Measuring Outcomes

• Developing a common set of rubrics for
  b.iWritten and Oral Communication
• Focus on the written dimension
• 3-5 commonly agreed upon elements
• Evaluation scale
• Divide into two groups
• A. Rubric for essays, reports, research papers
• B. Rubric for laboratory and scientific reports

32

• BREAK
• 20 minutes

33
Assessing Outcomes of the Core Curriculum

• Faculty-developed Institutional Outcomes of the
  Core Curriculum
• Written Oral Communication
• Aesthetic Sensibility
• Problem Solving
• Integration to the Campus the path to a
  lifelong, Lutheran-ethos learning environment
• Wholenessphysical, mental, spiritual
• Ethical Decision-making Moral Standards
• Historical Cultural Logic
• Cross-cultural sensitivity
• Servant Leadership

34
Assessing Outcomes of the Core Curriculum

• Collegiate Assessment of Academic Proficiency
  (CAAP) offers a program of standardized
  assessment to assess, evaluate, and enhance the
  outcome of general education
• 50 minute test modules in (multiple choice except
  essay writing)
• Writing skills
• Mathematics
• Critical Thinking
• Reading
• Science
• Essay writing

35
Assessing Outcomes of the Core Curriculum

• Advantages of CAAP over other choices
• Pick and choose modules
• Faculty can develop additional program specific
  questions
• Cost is reasonable
• Benchmark data is provided
• Option of choosing the sampling size
• For a sophomore class size of 140, sample size
  for a margin of error of /- 5 would be 100

36
Assessing Outcomes of the Core Curriculum

• Boundaries to consider
• When to give itintended for 2nd semester
  sophomores
• What modulescost is constant for 2 to 5 modules,
  provided same students take all modules
• Written essay module is an additive cost, and has
  the option of guided local scoring or ACT scoring
• How to administerhow to motivate students to do
  wellhow this impacts on the faculty

37
Assessing Outcomes of the Core Curriculum

• The first step in this process is planning for
  success
• Two part workshop process
• Part I
• Needs and assessment goals inventory
• Part II
• Evaluation of the importance of the areas in each
  of the 6 offered modules, aligned with the
  programs/courses in the core that cover this
  material

38
References

• ACT, (2006) Guide to Successfule General
  Education Outcomes Assessment 2006-2007.
• Barnes, E. M., et al., (2006). Curriculum
  Alignment for Student Achievement. Presentation
  at the SACS-COC Annual Meeting, Dec 2006. The
  presentation in PDF format and the audio
  recording of the workshop is available.
• Birnbaum, R. (2004). Speaking of Higher
  Education The Academics book of quotations.
  Westport, CT ACE/Praeger Publishers.
• Poe, R. E. and D. K. George, (2006). CS-87
  Evaluating Program Effectiveness Using Student
  Assessment Data Basic Steps for Beginners.
  Presentation at the SACS-COC Annual Meeting, Dec
  2006. Specific details associated with culling
  aggregated information from Comprehensive
  Exams,Internships, Essays, Portfolios, Oral
  presentations Juried Performances, Senior or
  Capstone Courses, and Embedded assessment
  available as is the audio recording of the
  workshop.
• SACS-COC (2005) Resource Manual (Available in
  hard copy and through the Concordia website under
  the Institutional Effectiveness and/or
  Accreditation 2008 pages posted in the Resources
  for Faculty Staff section.
• ALL RESOURCES AVAILABLE ON CONCORDIAs WEBSITE
• Under Resources for Faculty Staffselect
  ACCREDITATION 2008 or INSTITUTIONAL EFFECTIVENESS
  HOME