Technology does not

1
Academic Degree Program Design and Assessment
February 26, 2003

• Teresa Summers
• Bobby Matthews
• Karl Roider
• Carol ONeil
• David Cronrath
• Sarah Liggett

2
SACSCOC Reaffirmation of Accreditation

• Commission on Colleges of the Southern
  Association of Colleges and Schools (SACSCOC)
• New Principles of Accreditation
• No Self-Study
• No must Statements

3
Reaffirmation of Accreditation

• Based on strong culture of institutional
  INTEGRITY
• Will benefit LSU and the public
• Built on LSUs mission and purpose
• Includes student learning and systems and
  resources that support student learning
• Assumes a mature on-going planning and review
  process is in place

4
Reaffirmation of Accreditation

• Ensuring Institutional Effectiveness
• Documenting conditions
• Policies
• Planning processes
• Faculty credentials
• Financial well-being of the institution

5
Reaffirmation of Accreditation

• Reviewer expectations
• Statement of program goals and expected outcomes
• Plan for ongoing assessment of program goals,
  objectives and expected outcomes
• Evidence that programs and services have been
  assessed on a regular basis
• Evidence that the results of assessment have been
  used to improve programs, services and student
  learning outcomes

6
Primary components

• Compliance Certification
• Made up of Core Requirements Comprehensive
  Standards
• Attests to LSUs analysis of its compliance with
  accreditation requirements
• Due August 15, 2003

7
Primary components

• Quality Enhancement Plan
• QEP builds on established institutional planning
  and assessment process
• Course of action addressing 1 or more issues
  contributing to institutional quality
• Will include focus on student learning
• Due January 12, 2004

8
Timeline to Reaffirmation

• Off-site Review November 2003
• On-site Review March/April 2004
• For more information
• http//www.lsu.edu/sacscoc
• http//www.lsu.edu/planning
• http//www.lsu.edu/flagship

9
Planning Timeline By March 17

• measurable, time-bound performance indicators
  tied to goals/objectives
• information technology needs to support goals and
  objectives
• student learning outcomes and performance
  indicators for each curriculum
• resources needed to meet goals (human, financial,
  facilities, equipment, etc.)

10
Goals Associated with this Session

• To review process of assessing student learning
  in academic programs
• To revise assessment format as necessary
• To begin process of describing assessment thats
  been done
• To begin development of statement of how results
  of assessment are being used

11
Objectives Hierarchy

• Flagship Goals/Objectives
• Departmental Goals
• Program Learning Objectives
• Course Goals/Objectives

12
LSU Flagship Objective 1

• Increase research productivity and long-term
  economic development

13
General Departmental Goal

• Increase federally funded projects to 50 of
  total by 2005

14
Degree Program Learning Objective

• Students will be able to construct and evaluate
  arguments in light of historical evidence.

15
Course Learning Objective

• Students will be able to explain the logic
  underlying statistical hypothesis testing.

16
Assessment is . . .

• Systematic process of determining learning
  objectives
• Gathering and evaluating information about
  student learning outcomes (for the purpose of)
• Making decisions about programs

17
Student Learning Assessment in LSU Program Review
Criteria

• Explicit objectives (outcomes) for student
  learning
• Assessment criteria (performance indicators)
• Assessment processes
• Process for reviewing and revising the curriculum

18
Program Review Since 1995

• 150 degree programs
• 48 departments
• 2/3 of the way through

19
SACS Compliance Standard 16 The institution . .
.

• Identifies expected outcomes for its educational
  programs
• Assesses whether it achieves these outcomes
• Provides evidence of improvement based on
  analysis of those results

20
SACS Compliance Standard 1The institution
demonstrates that each educational program
establishes and evaluates program and learning
outcomes.
21
Learning Outcomes Assessment

• The use of data--gathered from valid assessment
  processes--for the purpose of determining the
  differences the program is making in its
  students.
• Involves faculty looking at student learning
  across a curriculum or program (rather than in a
  specific course)

22
Learning Outcomes Assessment involves

• Faculty discussion of goals and objectives for
  student learning
• Faculty discernment of how their courses
  contribute to changes in students in the
  direction of program goals
• Faculty development of greater program
  consciousness

23
Each course in the degree program should have

• Specific objectives for student learning that
  track program goals
• Assessment criteria and processes
• Both general and specific syllabi that indicate
  program goals, course learning objectives, course
  assessments with assessment criteria
• A systematic process through which faculty
  evaluate the effectiveness of the course

24
Definitions

• Goals broad statements of the general learning
  outcomes students should manifest by the end of
  the course
• Objectives specific, concrete measures or
  activities by which goals will be addressed
• Indicators express the observable level or
  extent of learning, change or gains

25

• Goal Students will develop critical insight
  into the influence of welfare regulations on the
  socio-economic status of 19th century English
  women.
• Objective Students will be able to describe the
  primary English welfare regulations and explain
  or exemplify how they specifically affected women
  in the mid-19th century.

26

• Performance indicators express the observable
  level learning or change
• Objective Students will be able to describe the
  primary English welfare regulations and explain
  or exemplify how they specifically women in the
  mid-19th century.
• Course Indicator Set of criteria based on
  levels of inclusiveness of the discussion
• Program Indicator The number of students who
  perform at each level (may want to specify a
  threshold of success at each level

27

• Explicit objectives for student learning
• http//www.jmu.edu/assessmemt/

28

• Q How can we know the degree to which we are
  meeting objectives of the curriculum?
• A Through regular and systematic collection of
  data about what students actually learned.

29
Examples of Assessment Methods

• http//www.umass.edu/oapa/assessment/program_based
  .pdf  
• Embedded assessment
• Capstone courses
• Senior assignments
• Interviews (I)
• Focus groups (I)
• Reflective essays
• Surveys (I)
• Objective Tests
• Case studies
• Classroom assignments
• Presentations

30
B.A. in History, LSU Program Goals

• To develop students awareness of the value of
  historical knowledge
• To assist students development of a sense of
  historical perspective
• To assist students acquisition and refinement of
  critical thinking, reading, writing
• To refine students ability to make and evaluate
  historical interpretations

31
B.A. in History Expected Learning Outcomes
Students will

• Understand change and continuity in history
• Be aware of the influence of varied and complex
  historical factors on the lives of individuals
  and societies
• Be able to examine a variety of primary and
  secondary historical sources critically and to
  analyze them in terms of their contexts
• Be able to construct and evaluate arguments in
  light of historical evidence

32

• Q How do we use assessment results?
• A (1) to adjust teaching approaches
• (2) to design new courses
• (3) to eliminate courses that dont meet
  the needs of our students

33
Foundation Knowledge and Skills for Didactic
Programs in Dietetics

• Communications
• Physical and Biological Sciences
• Social Sciences
• Research
• Food
• Nutrition
• Management
• Health Care Systems

These 8 topics are Subdivided further
Knowledge of Demonstrated ability to
34
CommunicationsKnowledge of

• Negotiation techniques
• Lay and technical writing
• Media presentations
• Interpersonal communication skills
• Counseling theory and methods
• Interviewing techniques
• Concepts of human and group dynamics
• Public speaking
• Educational materials development

Grid work of all classes in curriculum
35
Grid work of all classes in curriculum
CommunicationsDemonstrated ability to

• Use oral written communications in presenting
  an educational session to a group
• Counsel individuals on nutrition
• Document appropriately a variety of activities
• Explain a public policy position regarding
  dietetics
• Use current information technologies
• Work effectively as a team member

36
Communication Skills

• Public speaking (CMST 2060, HUEC 1021, 3016,
  3019, 4021)
• Educational materials development (HUEC 2019,
  3016, 4013, 4017, 4021)
• Use oral written communications in presenting
  an educational session to a group (1021, 2019,
  3016, 4021)

37
Use oral written communications in presenting
an educational session to a group (1021, 2019,
3016, 4021)

• HUEC 1021 a 1 minute talk poster to the class
• HUEC 2019 3016 community classes on nutrition
• HUEC 4021 25 minute PP presentation to the
  class or to dietitians in the community for CE
  credit

38
HUEC 3016 community classes on nutrition

• GUMBO 3016 students gave lessons on nutrition,
  food safety, and food preparation to homeless
  individuals to help them obtain entry level jobs
  in food service
• LSU students were evaluated by the instructor on
  the lesson plan and presentation
• LSU students were evaluated by GUMBO participants
  on effectiveness of the lessonusually pre-post
  tests
• LSU students evaluated the experience through a
  reflective journal
• The ultimate evaluation all but one participant
  failed to get a job in food service

39
School of Architecture
Louisiana State University
School of Architecture Assessment Environment

• Professional school accreditation (NAAB)
• Performance of graduates in field

40
School of Architecture
Louisiana State University
Performance of graduates in field

• Statistics from State Licensing Board
• Professional Advisory Board
• Alumni Survey
• Feedback from firms in the region

41
School of Architecture
Louisiana State University
Professional School Accreditation (NAAB)

• 37 performance criteria to be met with specific
  work products
• Examinations
• Projects
• Lectures
• Syllabus
• Correlate criteria with course work

42
School of Architecture
Louisiana State University
Professional School accreditation (NAAB)
Matrix displaying required courses and
criteria.
43
School of Architecture
Louisiana State University
Rebuilding the curriculum from the bottom

• Establish Course Objectives for Core Courses
• Faculty workshops on learning objectives
• Curriculum Committee Retreat with CELT
• Sharing information and examples
• Course portfolios (junior faculty)
• Identify Key Components of Curricular Design
• Develop Measurement Tools for Key Components

44
School of Architecture
Louisiana State University
Rebuilding the curriculum from the bottom
First Year 1001 Architectural Design I 6 1002
Arch Design II 6 1022 Math 3 1441 Math 3 1001
English Elective 3 1002 English 3 GE Social
Studies Elective 3 GE Social Science
Elec 3 Second Year 2001 Arch Design
III 6 2002 Arch Design IV 6 2003 Arch
Techniques 3 2006 Arch Topics IV 3 GE Natural
Science Elec 3 GE Humanities Elec 3 2001
Physics 3 GE Humanities Elec 3 GE
Humanities 3 GE Natural Science Elec 3 Third
Year 3001 Arch Design V 6 3002 Arch Design
VI 6 3003 Arch Structures I 3 3004 Arch
Structures II 3 3005 History of
Arch 3 3006 History of Arch II 3 3007 Arch
Systems 3 3008 Environ. Control Sys 3 GE Arts
(non-Arch) 3 xxxx Professional Elec. 3 Fourth
Year 4001 Arch Design VII 6 4002 Arch Design
VIII 6 4031 Arch Structures III 3 xxxx Arch.
Hist. III 3 4062 Urban Planning and
Design 3 5006 Professional Prac. 3 xxxx Profession
al Elec. 3 xxxx Professional Elec. 3 Fifth
Year 5001 Comprehensive Arch Design 6 5002 Arch
Des. 6 5005 Adv. Arch. Techniques 3 xxxx Professi
onal Elec. 3 xxxx Professional Elec. 3 xxxx Profes
sional Elec. 3 Free Elective 3 Free Elective 3  
1
2
Third Year Gate
3

• Portfolio review
• Review of GPA
• Review of GPA in major

4
5
45
School of Architecture
Louisiana State University
Rebuilding the curriculum from the bottom
First Year 1001 Architectural Design I 6 1002
Arch Design II 6 1022 Math 3 1441 Math 3 1001
English Elective 3 1002 English 3 GE Social
Studies Elective 3 GE Social Science
Elec 3 Second Year 2001 Arch Design
III 6 2002 Arch Design IV 6 2003 Arch
Techniques 3 2006 Arch Topics IV 3 GE Natural
Science Elec 3 GE Humanities Elec 3 2001
Physics 3 GE Humanities Elec 3 GE
Humanities 3 GE Natural Science Elec 3 Third
Year 3001 Arch Design V 6 3002 Arch Design
VI 6 3003 Arch Structures I 3 3004 Arch
Structures II 3 3005 History of
Arch 3 3006 History of Arch II 3 3007 Arch
Systems 3 3008 Environ. Control Sys 3 GE Arts
(non-Arch) 3 xxxx Professional Elec. 3 Fourth
Year 4001 Arch Design VII 6 4002 Arch Design
VIII 6 4031 Arch Structures III 3 xxxx Arch.
Hist. III 3 4062 Urban Planning and
Design 3 5006 Professional Prac. 3 xxxx Profession
al Elec. 3 xxxx Professional Elec. 3 Fifth
Year 5001 Comprehensive Arch Design 6 5002 Arch
Des. 6 5005 Adv. Arch. Techniques 3 xxxx Professi
onal Elec. 3 xxxx Professional Elec. 3 xxxx Profes
sional Elec. 3 Free Elective 3 Free Elective 3  
1
2
Critical thinking assessment
3
4
5
46
School of Architecture
Louisiana State University
Rebuilding the curriculum from the bottom
First Year 1001 Architectural Design I 6 1002
Arch Design II 6 1022 Math 3 1441 Math 3 1001
English Elective 3 1002 English 3 GE Social
Studies Elective 3 GE Social Science
Elec 3 Second Year 2001 Arch Design
III 6 2002 Arch Design IV 6 2003 Arch
Techniques 3 2006 Arch Topics IV 3 GE Natural
Science Elec 3 GE Humanities Elec 3 2001
Physics 3 GE Humanities Elec 3 GE
Humanities 3 GE Natural Science Elec 3 Third
Year 3001 Arch Design V 6 3002 Arch Design
VI 6 3003 Arch Structures I 3 3004 Arch
Structures II 3 3005 History of
Arch 3 3006 History of Arch II 3 3007 Arch
Systems 3 3008 Environ. Control Sys 3 GE Arts
(non-Arch) 3 xxxx Professional Elec. 3 Fourth
Year 4001 Arch Design VII 6 4002 Arch Design
VIII 6 4031 Arch Structures III 3 xxxx Arch.
Hist. III 3 4062 Urban Planning and
Design 3 5006 Professional Prac. 3 xxxx Profession
al Elec. 3 xxxx Professional Elec. 3 Fifth
Year 5001 Comprehensive Arch Design 6 5002 Arch
Des. 6 5005 Adv. Arch. Techniques 3 xxxx Professi
onal Elec. 3 xxxx Professional Elec. 3 xxxx Profes
sional Elec. 3 Free Elective 3 Free Elective 3  
1
2
Capstone project
3
4
5
47
School of Architecture
Louisiana State University
Next Steps

• Assess writing skills at various levels of the
  curriculum
• Assess representation skills

48
(No Transcript)
49
Assessment Measures Dept. of Accounting
50
SACS Standard 15 The institution

• Identifies competencies within the general
  education core
• Provides evidence that graduates have attained
  those college-level competencies

51
George Mason Universitys IT Across the
Curriculum Program

• http//www.cas.gmu.edu/tac/docs/it_goals.html

52
effective command of written English

• In effective writing, the writer accomplishes
  his or her purpose relative to the audience
  within a given context. In an academic context,
  effective writing is characterized by a clear
  purpose accurate, appropriate, and sufficiently
  developed content an organized and coherent
  structure and an appropriate voice. The writer
  follows the conventions of standard usage,
  mechanics, and spelling, and properly
  acknowledges and documents secondary source
  materials.

53
Primary Writing Assessment Methodology
Writing Portfolio Assessment
54
Ratings

• (1) end of 1st semester
• (2) end of 2nd semester (1st year)
• (3) end of 4th semester (2nd year)
• (4) end of 3rd year (6th semester)
• (5) end of 7th semester

55
4-Point Rating Scale

• 1 ineffective
• 2 marginal
• 3 effective
• 4 highly effective

56
Findings from Writing Assessments

• (1) At the end of 4 years, 65 of the students
  for whom we had a substantial portfolio were
  rated as effective writers.

57
Ratable Portfolios (n71)
Not Effective
Effective
58
Findings (cont) from Writing Assessments

• (2) In general, students writing did not improve
  over the course of their undergraduate studies.
• (3) Unless majoring in a writing-intensive
  discipline, students typically wrote few formal
  assignments beyond their first year.

59
Findings (cont) from Writing Assessments

• (4) After first-year English, students seldom
  received writing instruction or comments on
  graded written assignments that would lead to
  improved writing.

60

• Whats Next?