Program Outcomes and Measurement

1
Program Outcomes and Measurement

• QEP Non-Instructional Group

May 14, 2004
2
Narrowing the Focus

• Review of homework
• Validation results
• Finalization of issue

3
Narrowing the Focus

• Example of survey to narrow the focus (Guidance
  Counseling)
• Guidance and Counseling
• We want to serve you better! Please tell us
  about your experience with us.
• 1. Are you a (check one) ? new student
  ? returning student?
•  
• 2. What was the purpose of your visit to
  Guidance and Counseling today?
• ? to get a degree plan ? to register for
  classes ? to see a counselor about career options
• ? to talk about transferring to another
  institution ? a personal matter
• ? to discuss test scores ? other reason. Please
  explain_________________________
• 3. Did you get the information or assistance you
  needed?      ? Yes      ? No
• 4. How were you treated?  (Friendly, Courteous,
  Helpful, Informative)              
• ? Excellent           ? Good    ? Fair      ?
  Poor
• 5. What are your suggestions to help us better
  serve you?
• 6. Are you willing to be part of a discussion
  group to evaluate the Guidance and Counseling
  Department? If yes, please list your name,
  address and telephone number below.
• Name_________________________________________
  Telephone number________________
• Address__________________________________________
  ____City Zip_________________
• Other examples?

4
Now What???

• Use your narrowed focus issue to define next
  steps and write your Unit Action Plan
• Determine desired program outcomes that will
  improve your units performance related to your
  issue.
• Determine intervention method(s)what are you
  going to change to meet your desired outcome?
• Determine assessment technique.
• Develop timetable for implementation.
• Perform baseline measure.
• Use existing information from Institutional
  Effectiveness and Strategic Plan.

5
Write Expected Outcomes

• Step 1 Articulate the mission or purpose of
  your unit.
• What is the primary function of your unit?
• What core activities are involved?
• Who is your customer?
• What should those you serve experience after
  interacting with your unit?

6
Write Expected Outcomes

• Consider these questions
• What are the most important results or impacts
  that should occur as a result of your units
  activities?
• What are your critical work processes and how
  should they function?

7
Write Expected Outcomes

• Step 2 Define your units objectives.
• Objectives must be measurable
• Three categories of objectives
• Outcome statements
• Process statements
• Satisfaction statements

8
Write Expected Outcomes

• Outcome statements concern gains you want those
  you serve to make.
• What can someone do after interacting with your
  program?
• Example After their first visit to Guidance and
  Counseling, students should be ready to register
  for classes.

9
Write Expected Outcomes

• Process statements concern the accomplishments of
  your units functions.
• Level or volume of activity
• Efficiency with which you conduct the processes
• Compliance with external standards or regulations
• Example Students should wait no longer than 20
  minutes to be seen by a counselor.

10
Write Expected Outcomes

• Satisfaction statements describe how those you
  serve rate their satisfaction with your units
  processes or services.
• Example 80 of students report that their needs
  are met during their visit to Guidance
  Counseling.

11
Determine Assessment Techniques

• Step 3 Determine appropriate assessment
  measures and criteria (how will you prove it?)
• Define and identify the sources of evidence you
  will use to determine whether you are achieving
  expected outcomes.
• Detail what will be measured and how it will be
  measured.
• Create measures that help your unit in making
  critical decisions about its processes and
  services.
• Use multiple measures.

12
Determine Assessment Techniques

• Criteria or targets for success Always aim for
  a level that stretches your units performance.
• How well should we serve our clients?
• 95 of our users will be very satisfied or
  satisfied with our services.
• At least 80 of eligible employees will
  participate in training.
• 90 of the transcripts will be sent within three
  days.
• 90 of the forms will be processed without errors.

13
Determine Assessment Techniques

• Common types of assessment
• Attitudinal measures of satisfaction from those
  you serve
• Direct counts of unit services
• External validation (neutral party, auditor,
  professional standards)
• Control for any variations that may occur
  baseline and follow-up must measure similar
  people/process/outcome.

14
Determine Assessment Techniques

• The type of objectives, or desired outcomes, you
  set will determine what type of assessment you
  use.

15
Determine Intervention Method

• Step 4 Determine intervention method.
• What changes can you implement to reach your
  target?
• Keep it simple making too many changes at once
  will complicate measurement (you wont know what
  worked!)
• Typical types of change include
• Revising organizational structure
• Reallocating resources
• Revamping administrative procedures/processes
  (personnel, systems, automation)
• Modifying or expanding relations with public or
  external agencies

16
Develop Timetable

• Late summer/early fall Establish a baseline
  (current performance)
• Spring Implement intervention
• End of Spring/Summer Re-assess
• Timeline will depend on what is being measured
  and how often the process occurs
• For example, registration efficiency can only be
  measured during registration, so baseline would
  be established in the fall, intervention
  implemented in spring, and re-assessment in
  Summer and the following fall.

17
Next Steps

• Review Guide for QEP Non-Instructional
  Specialists
• Ensure that focus area is narrowed
• Begin to fill out Action Plan and Assessment
  Report Template
• Determine intervention
• Determine assessment technique
• Establish Timeline
• Call your non-instructional team with any
  questions.
• Submit template online no later than August 31.
• Follow-up training will be a fall workshop on
  baseline assessment.

18
Our Slogan

• If youre not serving a student, youre serving
  someone who is!