Collecting Automated Customer Feedback

1
Collecting Automated Customer Feedback

• July 2007

2
Objectives of Webinar

• Provide you with information that will assist you
  in launching an Automated Customer Feedback
  Survey.
• Overcoming Challenges.
• Help you build stronger brand loyalty through
  your Contact Center.

3
Project Team

• Center Director
• Director of Training and Quality
• Corporate Senior HR Manager
• Senior IT Manager
• Multimedia Specialist
• Development Coach

4
Objectives

• Incorporate direct customer scores from the IVR
  Guest Survey into agent performance.
• Agent Merit Program (monthly yearly
  evaluations).
• Commitment To Excellence Program.
• Reduce DC call monitoring by 50, thus providing
  more time to coach develop.

5
Project Assumptions

• Customer feedback scores will be available for
  agents to review.
• Survey scores will be worth 50 of agent quality
  performance rating.
• Reduce call monitoring by 50.
• Agent scorecard/CONNECT will be modified to
  reflect new merit calculations.

6
IVR Guest Survey Questions

• Question 1
• How would you rate the helpfulness of your
  Reservations Sales Specialist?
• Question 2
• How would you rate your confidence in the
  accuracy of the information our Reservations
  Sales Specialist gathered and provided to you?
• Question 3
• When you do call us to make a reservation again,
  how pleased would you be to speak with the same
  Reservations Sales Specialist that helped you
  today?

7
Overview

• Current Process
• QA is comprised of two monthly monitoring
  sessions, which include three calls for each
  session.
• This metric is 100 points or 25 of an agents
  annual merit.
• 2005 Process
• New Quality Performance section that comprises QA
  Guest Survey Score (GSS).
• Beginning January 1, 2005, all agents will
  receive one QA monitoring session, and up to 5
  Automated Guest Surveys per month.
• The two scores will be weighted as follows
• ATP Monitor Score (60 of Quality Performance
  Score) 15 of agent merit.
• Guest Survey Score (40 of Quality Performance
  Score) 10 of agent merit.

8
The Process

• Each RSS will receive 5 Automated Guest Survey
  Scores (GSS) per month.
• The individual GSS scores will be loaded into
  Connect and the Agent Scorecard so the agent can
  view their month-to-date progress. The GSS
  scores will be averaged and an aggregate score
  will be determined.
• The GSS score is then assigned a point value (see
  point grid on next slide). The GSS point value
  and the QA point value are added together to
  determine the overall Quality Performance score.
• The RSS receives a monthly/yearly evaluation from
  their DC. Their performance (including their GSS
  score) is reviewed at that time.
• The new GSS will also be incorporated as a
  qualifying factor for the CTE Award goal for
  GSS will be 95.

9
QA GSS Scoring Grids

• QA Scoring Grid

GSS Scoring Grid
10
Scoring Method (Today)
11
Scoring Method (2005)
12
The Pilot

• We conducted a one month pilot during October to
  test our assumptions and determine if we needed
  to adjust our plan.
• The Tampa CRO was the test site for the pilot.
• We compared our current process and the new
  process simultaneously to determine the
  effectiveness and fairness of the new program.
• The results of the Pilot were analyzed, and
  shared with the HRW Executive Committee.

13
The Pilot (Results)

• Average scores attained during pilot
• The average ATP Score during the pilot was 94.3
• The average GSS score during the pilot was 95.6
• Average number of points earned during pilot
• Old Scoring Method 84.2 points (100 possible).
• 2005 Scoring Method 84.4 points (100 possible).

14
Pilot Results Cont.

• First Analysis
• Total Comments 181
• Comments not specific to agent 17
• of comments not agent specific 9.4
• Second Analysis
• Total Comments 121
• Comments not specific to agent 7
• of comments not agent specific 5.8

15
The Pilot (Technology)

• The technology developed for this project has
  been very successful. We have received positive
  comments about the changes in Connect and the
  Agent Scorecard.
• Connect
• The new GSS metric has been fully integrated into
  Connect. The agents and DCs are able to access
  their GSS scores as the surveys are completed.
  In addition, the agents are able to see their
  month to date scores, get a breakdown of the
  individual surveys, receive a breakdown of each
  question in the survey, and listen to any
  comments left by their guests.

16
The Pilot (Technology)

• IVR
• The Horizon 1.12 enhancement has enabled us to
  more efficiently and effectively capture surveys
  on the agents.
• Agent Scorecard
• The new GSS metric has also been integrated into
  the Agent Scorecard. The agents are able to view
  their month to date GSS scores on the home page,
  and can determine their overall Quality
  Performance score on the merit screen of the
  scorecard. The scorecard also includes a
  pop-up box that shows the scoring grid that is
  utilized for the new metric.

17
Project Concerns Solutions

• Concern
• Will we be able to capture 5 surveys on each RSS
  each month?
• Solution
• During our 28 day pilot period, 95 of all Tampa
  agents received the maximum 5 surveys. 98 of
  the agents received at least 3 surveys which is
  our minimum threshold. We felt confident that
  the new Horizon enhancement has enabled us to
  achieve our goal.

18
Project Concerns Solutions

• Concern
• Agents are concerned that guests are scoring them
  on their experience with the hotel or Hilton
  overall.
• Solution
• We have modified the opening of the automated
  survey to ensure the guest knows they are rating
  their experience with the RSS that handled their
  reservation. Further enhancements to the IVR
  Survey are being implemented in January to
  capture additional guest feedback

19
Project Concerns Solutions

• Concern
• Agents are concerned that the automated guest
  survey will not be reflective of their overall
  performance.
• Solution
• Our goal is to capture 5 automated guest surveys
  per month for each agent in the Network. We will
  capture no more than one survey per day per agent
  to provide additional randomness. This should
  ensure that we receive a fair representation of
  the agents performance.

20
Project Concerns Solutions

• Concern
• What if agent doesnt receive 5 automated guest
  surveys?
• Solution
• A Horizon enhancement (1.12) enables us to count
  the number of surveys each RSS receives, which
  should assist us in meeting our overall goal of 5
  per RSS. However, our minimum will be 3. If an
  RSS receives less than 3 surveys, then this
  portion of their Guest Service Score will not
  count towards their performance.

21
Project Concerns Solutions

• Concern
• Agents are concerned that a guest may incorrectly
  score the automated guest survey (i.e. based on
  the transcribed comments, we know that the guest
  was happy with the agents performance, yet they
  gave the agent a 1,1,1 score).
• Solution
• An individual score can be thrown out if there
  is clear evidence from the recorded comments that
  the guest scored the survey incorrectly. All
  guests who score gives all 1s are contacted by
  the agents DC.

22
Project Concerns Solutions

• Question
• Agents who receive a 1,1,1 score, often do not
  receive written comments to substantiate the
  reason for the low score.
• Answer
• We have implemented a new procedure that will
  enable us to obtain further feedback about
  surveys with a 1,1,1 score. Every Tuesday and
  Friday, a report will be published that lists all
  of these surveys by CRO. Each of the CROs are
  responsible for contacting the guests to
  understand why they rated the agent so low. The
  guests feedback will be shared with the agent.

23
Project Concerns Solutions

• Concern
• Agents may not confirm to IVR if they have a bad
  call or a guest that is upset.
• Solution
• We believe that the majority of agents will
  follow their normal practice of confirming to
  IVR. We have a check balance with DC
  Monitored calls. If necessary, we can utilize
  Witness if the agent consistently falls short of
  the 5 surveys. A monthly report is generated
  that shows the percentage of calls each agent
  transferred to the IVR.

24
Project Concerns Solutions

• Concern
• Agents are concerned that the automated guest
  survey will significantly impact their merit and
  ability to earn CTE.
• Solution
• The new Quality Performance Score, which will
  incorporate the QA Score and the Automated Guest
  Survey Score be weighted as follows
• QA Monitoring Score (60)
• Guest Survey Score (40)

25
Project Concerns Solutions

• Question
• Can I receive discipline if I fall below a
  certain GSS performance level?
• Answer
• We do not discipline on the raw score. We do
  follow progressive discipline based on a guests
  recorded comments and conversations held with a
  guest.

26
Quality Performance Score

• Guest Survey Score (40)
• QA/Monitored Calls (60)
• Quality Performance Score
• The Automated Guest Survey Score was given a
  smaller weight in relation to the QA/Monitored
  Calls because there is no back-up to support the
  score. It can not be verified unless the guest
  leaves a recorded comment. The QA/Monitored
  Calls can be verified through Witness.

27
Major Challenge

• Acceptance of this new metric has been difficult.
• Agent perceptions are varied, but we encountered
  resistance during the pilot.
• There are very few comments left by guests to
  back-up the scores.
• Perception from agents is that the GSS scores are
  much lower than the ATP scores the opposite is
  actually the truth.
• Selling the importance of this new metric is key.

28
Review Objectives of Webinar

• Provide you with information that will assist you
  in launching an Automated Customer Feedback
  Survey.
• You have seen the process we used and I hope it
  will be helpful in your decision to implement an
  Automated Customer Feedback Survey.

29
Overcoming Challenges

• All survey that receives all 1s are automatically
  called back.
• Surveys where the comments do not match the score
  are called back.
• All positive guest comments are transcribe,
  signed by the director and posted. The top five
  comments receive up to 50 points towards our
  agent recognition program.
• No disciplines based on raw scores.
• Newer agents do not have the same distrust with
  the Automated Survey and look forward to hearing
  the guest comment and seeing their scores.
• Agents have the ability to see scores and listen
  to comment for the previous day, which relieves
  some anxiety.

30
Help you build stronger brand loyalty through
your Contact Center.

• The survey is an easy way for guest to provide
  feedback.
• Our following up on good and bad calls was a big
  hit with guest.
• Ability to respond to guest issues in an
  expeditious manner when the guest is not
  expecting it.
• Encourage guest to take survey and leave feedback
  in the future.

31
Where are we now?

• Capture 11 surveys per agent per month.
• The lowest score for each agent is dropped.
• Retained both monitoring sessions.
• Changed method of scoring.
• Increased number of survey questions to 5 and
  rating scale of 1 to 5.

32
Recent Success Story

• UK Desk Automated Guest Survey increased by 8.9
  points from May 2007 July 2007.

33
Questions?