Insights from customer needs and satisfaction at UCF

1
Insights from customer needs and satisfaction at
UCFs FEEDS program

• Results from the Voice of the Customer component
  of a Six Sigma methodology project.

Roberto Champney1 Alain Gaumier2 Naomi
Morris1 1University of Central Florida 2Optimum
Consulting, Inc.
2
Objectives

• Project Background and Description
• Six-Sigma methodology overview
• Voice of the Customer (VOC)
• Results and Insights From VOC
• Recommendations

3
Project Background

• FEEDS- Florida Engineering Education Delivery
  System.
• Created in 1982 by the Florida State Legislature
• The system has evolved from a tape system in 1982
  to a CD and web-based education option in its
  current form.
• The University of Central Florida (UCF) is one of
  12 universities in the state that participates in
  the program, and holds the largest enrollment,
  38 (2002-2003).

4
Project Background

• ASQ (American Society For Quality) Community Good
  Works Project.
• Improve communities through the use of quality
  tools and technologies.
• Provides skills transfer to non-traditional
  quality practitioners.
• Project Goals
• Document the fall 2003 version of the FEEDS
  system.
• Identify Priority Process Improvements.

5

Six-Sigma methodology overviewDMAIC
6
Classical View of Performance

• Practical Meaning of 99 Good
• 20,000 lost articles of mail per hour
• 15 minutes of unsafe drinking water each day
• 5,000 incorrect surgical operations per week
• 2 short or long landings at most major airports
  each day
• 200,000 wrong drug prescriptions each year
• No electricity for almost 7 hours each month

7
Voice of the Customer (VOC)

• Technically there are numerous ways to enhance
  the system, yet there has been little attempt to
  focus this enhancement to represent the needs of
  its users.
• Identify customers and their needs.
• Collect and analyze reactive data.
• Consider proactive approaches.
• Convert collected data into customer needs.
• Sort out the most important attributes ctq.
• Obtain specifications for the critical-to-quality
  characteristics.

8
Voice of the Customer (VOC)

• Conducted a student focus group.
• Conducted several interviews with FEEDS students
  to
• Capture major needs categories and issues.
• Further understand FEEDS users.
• Developed a survey concerning the satisfaction
  and needs of FEEDS student users.
• End of Semester FEEDS Student Online Survey.
• Set adequate performance levels.
• Correlations to identify key drivers.
• Align opportunities for improvement with FEEDS
  UCF priorities.

9
The Surveys

• Two surveys over a two semester period
• Interview comments were used to create the survey
  questions
• 30 and 39 questions
• Some demographics.
• gender, study program, program level.
• Three types of questions (5-point likert scale
  and open ended).
• Satisfaction with current attributes of the
  system.
• Usefulness and Importance of particular
  attributes of the system (hypothetical and
  current).
• Comments on particular issues with the system.

10
The Survey (Sample Questions)

• Overall how satisfied are you with the FEEDS
  system?   
• Overall how effective are FEEDS courses compared
  to live courses?
• How satisfied are you with the quality of the
  video stream (vision and audio) on your FEEDS
  courses?
• Audio
• Image
• How satisfied are you with the communication with
  FEEDS professors?

11
The Survey (Validating Sample)

• Response Rate 338 (1st survey) and (149 2nd
  Survey)
• Mass email, and site banner.
• Percentages represent actual FEEDS student body
  distribution

12
Insights From VOC
13
Insights From VOC Overall Satisfaction
Comments69 and 54 of respondents gave
comments

• First Survey
• 30 positive comments
• 41 negative comments
• Second Survey
• 39 positive comments
• 19 negative comments

Note one respondent may have contributed to more
than one category of comments.
14
Insights From VOC

• On Spring 04 UCF FEEDS upgraded their stream
  format.

15
Insights From VOC

• Prioritize future efforts using differences
  between received and desired
• See SERVQUAL survey method.

Delivery of Service Quality," Journal of
Marketing, April 1988, pp. 35-48. Parasuraman,
Zeithaml and Berry, "SERVQUAL A Multiple-Item
Scale for Measuring Customer Perceptions of
Service Quality," Journal of Retailing, Spring
1988, pp. 12-40
16
Insights From VOC

• How often are you able to attend a live class?
• 29 to 39 always (4 5 scores)
• For what reasons are you not able to ALWAYS
  attend a live class?

17
Insights From VOC

• Please indicate all the mediums with which you
  communicate with FEEDS professors? (Check the two
  most frequent)

18
Insights From VOC

• How long does it takes on average to get a
  response from your professor?
• How satisfied are you with how long it takes to
  get a response from your professor?
• 55 satisfied (45 score)

19
Insights From VOC
In the event that you could have access to the
class notes before the class session, what would
be the best way to gain access to them?

• Where do you regularly watch the
• FEEDS stream or CD? (Check the
• two most frequent)
• Home 126
• Work 47
• Library 17
• Computer Lab 18
• Other 8
• (Beach, Starbucks, Outdoors)

20
Insights From VOC

• How often have you experienced missing material
  from a FEEDS stream?
• In the event that you have experienced missing
  material from a FEEDS stream, what part was
  missing?

21
VOC Instrument Recommendations

• 5 interviews will identify most of the problems.
  
• Dont get carried away with questions, youll
  have to make sense of them later.
• Everything is important or useful.
• Be careful with these types of questions.
• Ex. Teacher assistant meetings
• Usefulness score 4.07
• Attend score 2.79

22
VOC Instrument Recommendations

• Open ended comments are most valuable, but very
  time consuming.
• Use at least two persons for categorization.
• Provide enough space for comments.
• 255 characters is not enough
• Use online surveys, dont send attachment surveys

23
Insight Summary

• Identify Student Strata
• 100 remote students.
• Miss student interaction.
• Miss out on after class talk.
• Value recording of pre-class and break
  periods
• Convenience students
• Register for FEEDS but attend live classes.
• Use FEEDS as backup or study aid.
• Gender differences
• Females more than males value the interaction
  with other students and professors (per
  comments).
• Study Program differences
• Teacher Assistant meetings, and problem solving
  groups.

24
Insight Summary

• User satisfaction and system effectiveness are
  correlated.
• Effectiveness is driving satisfaction.
• Communication with professors is critical to
  satisfaction and effectiveness.
• 1/3 of students regularly attend live classes
• Email and in-person communication is used mostly
  for communication between FEEDS students and
  professors.

25
Insight Summary

• Provide access to class notes before class, when
  possible.
• Consider scanning professor hand notes if
  possible.
• Chalk/White Board is not appropriate for this
  medium.
• Live students should see the same image as Remote
  students.
• Theyll become the directors of the class
  production.

26
Insight Summary

• Standardize equipment and training.
• Encourage (enforce) professor training.
• Consider standardizing method of delivery and
  procedures (i.e. all use WebCT, all email
  assignments).
• Use one location for files, content, or materials
  access.
• Need commitment from professors.

27
Insight Summary

• Computer demos are particularly difficult to
  follow (unable to read buttons and options).
• Most of students problems with the stream are
  with accessing it due to technical difficulties.
• Provide content online, consider avoiding
  mailings (they consume resources).

28
Insight Summary

• Other online services or university sections
  confused as part of FEEDS (i.e. WebCT).
• Apply VOC methodology with other stakeholders
• Professors, Administrators, etc.

29
Questions ?