A StudentCentered Feedback Control Model of the Educational Process

1
A Student-Centered Feedback Control Model of the
Educational Process

• Molly Shor Robby Robson
• Dept. of ECE Dept. of Math
• Oregon State University
• Corvallis, OR 97331

2
Goal investigate new paradigm for
continuous-improvement

• Current continuous-improvement processes are
  based (loosely) on SPC - examining aggregate
  performance of students - and modifying the
  program
• We investigate a continuous-improvement process
  based on feedback control - examining individual
  performance of student and modifying students
  learning experiences

3
Old ABET

• Evaluation of Program
• Do Program Courses Cover Required Knowledge
  Skills?
• Students Jump Program Hurdles (Courses) with
  Satisfactory Grades
• Grades Measure Design, Technical Knowledge
  Problem-Solving Skills

• Evaluation/Improvement Cycle every 3 to 6 years

4
ABET 2000

• Continuous Improvement Process for Program Using
  Assessment of Student Learning

Program Creates Student Learning
Program

• Student Learning
• Interpret Data
• Formulate Problems
• Design Systems
• Understand Impact
• Ethics
• Communication

Assessment of Student Learning Feedback Changes
Program
5
Program-Centered Process Evaluation of
Aggregate Skills
loop external to program

• Feed back
• improves program
• improves teaching
• affects later students
• Feed forward
• remediates for current students

loops internal to program
6
Statistical Process Control Program-Centered
Feedback
Program
entering students
graduates

• Advantages
• Determines if process is producing desired
  results and when process has changed
• Statistical analysis is simple and cheap
• Some assurance that graduates perform adequately,
  on average, for constituents

7
Statistical Process Control Program-Centered
Feedback

• Disadvantages
• Statistical process control indicates whether
  there is a problem, not how to fix it (try
  something different)

8
Statistical Process Control Program-Centered
Feedback

• Disadvantages
• SPC tries to ensure a constant mean and variation
  in process outputs. The assumption is that the
  process causes any variation in outputs.
• The inputs to our process (entering students)
  vary significantly, and their variation is not
  controllable as part of the process

9
Statistical Process Control Program-Centered
Feedback

• Disadvantages
• One of our goals is that all graduates meet or
  exceed minimal expectations in all of a set of
  skills
• Suppose performance of our graduates is normally
  distributed and we are able to control its
  average. We would have to set average
  performance over 2.3 standard deviations above
  required performance to ensure that 99 of
  students meet minimal standards.
• Average students A to guarantee poorest
  student C.

10
Student-Focused Process Evaluation of
Individual Skills (focus on desired outcomes)
Informed of outcomes - student buy in -
Graduation based on satisfying outcomes
Project and course evaluation based on outcomes
Documentation of students progress toward
outcomes
Educational experiences adjusted based on
students progress towards outcomes
Advised on progress toward outcomes
11
Student-Focused Process

• Initial Condition initial knowledge, skills,
  assumptions background of student
• Desired Output set of program outcomes, or
  course learning objectives
• Goal guarantee that each student in a diverse
  class of entering students will meet all program
  outcomes

12
Feedback Control Student-Focused Process

• Goal guarantee that a process with varying
  initial conditions will have a consistent output
• Mechanism feedback control
• Advantages reduces sensitivity of outcomes to
  variations in process parameters, initial
  conditions and disturbances

13
Open Loop System

• Educational program in which state of students
  achievement with regard to program outcomes is
  not used to adjust the sequence or nature of the
  students educational experiences
• Example traditional engineering program with
  prescribed sequence of classes

versus Closed Loop System

• As student progresses through program, close the
  loop around the student
• Measure the students progress on outcomes
• Use results to adjust the learning experiences
  for the student

14
Closing the LoopBest Practices

• Identification articulation of desired
  objectives outcomes
• Intelligible to all - students, instructors,
  program administrators
• Express so students know if they understand or
  not
• understand the first law of thermodynamics is
  open to interpretation by students who do not
  know a priori what is required
• State and illustrate the first law of
  thermodynamics (i.e., the conservation of mass
  and energy) and its basic concepts including
  conservation of energy within a system between
  kinetic energy, potential energy and internal
  energy and transfer of energy from the
  surroundings to the system by heat, work and flow
  of mass.

15
Closing the LoopBest Practices

• Assessment of outcomes
• Assess student progress toward learning
  objectives
• Measure process skills directly
• Measuring process skills must be accompanied by
• assessment criteria matched to desired outcomes
• preserving data on individual outcomes, rather
  than lumping into single cumulative grade
• reporting results of assessment to students

16
Closing the LoopBest Practices

• Feedback to the student
• Connect actual output to desired output by
  meaningful and frequent reporting to students
• make students more aware of which skills they
  need to learn
• use formative forms of feedback, focused on
  output variables and level of performance on
  outcomes
• scoring guides (or rubrics)
• Objective 1 2 3 a b e
• Test 1 2 2 1 2 1 2
• Project 1 3 3 1 2 2 2
• Project 2 4 3 3 3 2 3
• draft revision cycle
• peer assessment
• adaptive self-assessment tests

lt 3 - progress needed 3 - meets min. desired
level gt 3 - above min. desired level
17
Closing the LoopBest Practices

• Control methods
• Alter the educational experiences of a student in
  response to feedback provided by assessments
• Instructional techniques
• provide feedback to students to increase
  awareness, so students strive towards known,
  well-defined goals
• allow for differences among students give more
  attention to areas in which students need the
  most assistance
• methods include
• group work
• cooperative learning
• matching projects to students background

18
Closing the LoopBest Practices

• Control methods
• Alter the educational experiences of a student in
  response to feedback provided by assessments
• Academic advising
• map learning objectives to courses use progress
  on outcomes to plan future courses
• better and more granular information about a
  students skills is needed
• advisors may evaluate students knowledge and
  skills
• instructors may provide feedback on student skills

19
Program-Centered vs. Student-Centered
Continuous-Improvement Processes

• Program-Centered Process
• determine whether existing learning experiences
  should be added, deleted, or modified from
  program
• Student-Centered Process
• help diverse group of students reach the desired
  program outcomes