Carlton.com Website Assessment Visitor Satisfaction Study

1
Carlton.com Website AssessmentVisitor
Satisfaction Study

• Quantitative Attitude Assessment and Tracking of
  Visitors to Carlton.com
• DEMO REPORT

April 1, 2008
1
2
Objectives

• This initial report is to assess the value of the
  instrument in
• Capturing accurate data
• Monitoring which other travel and lodging sites
  our customers are using.
• Understanding purpose of site visit.
• Understanding customer expectations before they
  use the site.
• Measuring ongoing satisfaction with Carlton.com
  relative to features and capabilities on the
  website and relative to customer expectations and
  experience on other sites
• Identifying key drivers of satisfaction and
  willingness to recommend.
• Monitoring barriers to making a reservation.
• Identifying customer profile on Carlton.com.
• Providing information on acceptability of
  performance of the website for different segments
  of the visitor population
• Very Heavy Users
• Frequent Users
• Infrequent and New Users
• Rewards Tiers
• Transactors
• Information Seekers
• Promotion respondents

3

• Survey invitation delivered randomly to 1 out of
  every 100 visitors to Carlton.com homepage.
  Invite box is built into the home page instead of
  a pop-up (to avoid bias from customers who use
  pop-up blockers).
• They either dismissed the invitation, or if
  clicked yes, were asked a few preliminary
  questions then were asked to complete the
  interview after their visit was over.
• Visitors are not eligible to receive a second
  invitation until 90 days after receiving their
  first (unless they delete their cookies).

4
Methodology - 1

• The pre-questions established and rated their two
  favorite non-Carlton.com travel or lodging
  websites and their expectation level for their
  visit to Carlton.com today. After completing
  their transactions on the site, they were asked
  to rate their actual experience with the site
  today compared to their expectation.
• Respondents then rated the website on four key
  dimensions Ease of use, How quickly they were
  able to find what they wanted, How they would
  rate their experience overall from Wonderful to
  Painful and whether they would Recommend
  Carlton.com to friends or colleagues.
• A Linescore was immediately calculated and shown
  to the respondent for their agreement or
  disagreement, then verbatim reasons for their
  rating were recorded. In addition, they checked
  off up to five main reasons for rating from a
  list of positive and negative possible reasons.
  

5
Methodology - 2

• Activities they did or wanted to do today were
  selected from a list of possible information or
  transaction. The selected activities were then
  randomly presented for rating and comparison on a
  Linescale anchored by their personal overall
  rating of the website plus their prior ratings of
  each activity previously rated.
• Other information was gathered including
• Whether they successfully made a reservation, why
  not if not, and what they planned to do instead
• How they arrived at the site search engine,
  bookmark, email promotion, etc.
• Frequency of travel
• Frequency of visits to the site
• Reasons for their visit
• Rewards Member status
• Carlton hotels frequented past year
• Personal demographics
• Rating of the interview itself
• 582 completed interviews by unique visitors 74
  completion rate

6
Overall Rating of Carlton.com by visitors
Preference Segments Linescale segments
individuals into four categories based on a
scientific algorithm score derived from each
respondents rating versus competitive favorites,
expectation for the site, actual experience today
versus expectation and four key diagnostic
measures (Ease, Speed, Pleasant,
Recommendation). Acceptors are strong endorsers,
satisfied in all respects Borderline are either
missing a strong benefit or there is a
negative Indifferent are generally not
satisfied Rejectors are very dissatisfied
7
Overall Acceptance of Carlton.com - LINESCORE

• Carlton.com scored 56 Acceptors during
  February, 2008. An excellent score. In top 10
  of all Transaction websites tested to date.
• Indifferent and Rejectors totaled 19
• Expectation for Carlton.com was at 75 over
  favorite non-Carlton.com site
• Experience was rated higher than favorite
  non-Carlton.com site by 71

8
Overall Linescale Ratings Total Sample

• The Expectation for Carlton.com is at a very
  high level, and the Experience, while also very
  high, is slightly lower than Expectation.
• Both Carlton.com Expectation and Experience
  greatly exceed rating of competition.
• These are mean ratings for the aggregated
  Favorite non-Carlton.com travel website, the
  aggregated Next favorite travel site, the number
  1 favorite (Hilton.com), the number 2 favorite
  (Expedia.com), the number 3 favorite
  (Marriott.com), the Expectation for Carlton.com
  and the actual Experience today with Carlton.com.
  

9
Overall Linescale Ratings - Acceptors

• The Acceptor segments Expectation is higher
  than Total Sample, but Experience today exceeds
  the very high Expectation, as well as competition.

10
Reasons for Rating Drivers contributing to Score

• Average visitor checks 5.84 reasons
• TOP REASONS Overall and for Acceptors
• Has all the information I want
• Very fast in general
• Lets me make a reservation easily
• Organized in a logical way
• Key negatives are
• Not that easy to find exactly what I wanted
• Too many steps
• Pages and layout too crowded
• Did not have exactly the information I wanted
• Didnt show me what I wanted to see

11
Discriminate Reasons For Acceptance and Rejection

• Differences in Reasons Why Acceptors rated
  Carlton.com differently from Borderline,
  Indifferent and Rejectors are Drivers which
  influence satisfaction.
• Differences in rating of Activities done or
  intended are also Drivers of preference.
• This data can be useful to developers and
  marketers in improving satisfaction with this
  service point
• Analysis is available including individual
  verbatim comments on why visitors rated as they
  did.

12
Difference Scores Acceptor vs. Rejector
Key differences in drivers between Acceptors and
Rejectors are having all the information they
wanted and speed in general. Key negatives are
difficulty in finding exactly the information
they wanted, too many steps to get what they
needed and pages and layout too crowded.
13
Difference Scores Acceptor vs. Indifferent
Indifferent thought it was not easy to find
exactly what they wanted and did not have exactly
the information they wanted, although the
information may have been there. Too many steps
to get what I needed is also cited. Pages and
layout crowding less of a factor than for
Rejectors.
14
Difference Scores Acceptor vs. Borderline
Borderlines have similar positives to Acceptors,
but with less enthusiasm. Borderlines do not have
strong negatives.
15
Activities Intended or Done on the Visit

• The average visitor does or intends to do 4.9
  activities
• Check Rates (70) and Check Carlton Rewards
  point levels (58) are the two dominant
  information activities
• Intention to Make a Reservation at a Specific
  Hotel (51) and to View, Cancel or Modify my
  Reservation (44) are the two dominant
  transactional activities
• Viewing Maps and Area Information (38) and
  Viewing Hotel Photos (37) are heavily used
  information activities
• Visitors who are exploring for information are
  more likely to be Rejectors.

16
Ratings of Activities by All Visitors

• Transactional activities and viewing Reward
  balance are very highly rated
• Room for improvement in informational
  activities

Activities done during 2 or less visits not shown
17
Rating of Activities by Acceptors Only

• Acceptor pattern of preference is similar, but
  at a higher level
• Acceptors, too, show room for improvement in
  some informational activities

Activities done during 2 or less visits not shown
18
Rating of Activities Borderline
19
Rating of Activities Indifferent
20
Rating of Activities Rejectors
21
The Competitive Favorite Travel or Lodging Sites

• Hilton.com is by far (18) the favorite
  competitive hotel site
• Starwood.com (5), Holiday-Inn.com (5),
  Kayak.com (4) and Hotels.com (3) are the other
  significant hotel sites used by Marrriott.com
  visitors

Acceptor Index indicates above average Acceptance
of Carlton.com by those whose non-Carlton
favorite is the competitor site
22
Frequency of Visit and Rewards Membership

• Fifty-Seven (56) of visits are by visitors who
  use Carlton.com either a couple of times a week
  or every day. This will over-state the impact of
  frequent visitors. (There can be only one first
  time visit there can be hundreds of repeat
  visits per year.)
• Eleven percent of visits (11) are by new or
  infrequent visitors
• Ninety-Seven percent (98) of visits are by
  Rewards members
• First-timers and a couple of times a month
  are most likely to be Indifferent or Rejectors
• Non-Rewards Members are most likely to be
  Indifferent or Rejectors
• A conclusion is the site appears somewhat more
  geared to Heavy Users and Elite Rewards Members
  who know how to find what they want easily

23
Preference Segments by Rewards Status

• Acceptance increases or decreases as does
  rewards status
• Higher status levels also experience
  Indifferent and Rejectors

Non-Rewards Member
Elite Rewards Member
Presidents Club Rewards Member
Platinum Club Rewards Member
Black Card Rewards Member
24
Preference Segments by Visit Frequency

• Acceptance increases or decreases as does
  Familiarity
• Higher usage levels also experience Indifferent
  and Rejectors
• Even daily visitors have issues

Visits, not visitors
Light User This is my first time or A few times
a year Moderate User A couple of times a
month Heavy User A couple of times a week Daily
User Just about every day
25
Top Reasons for Rating by Rejector Daily Visitors

• Daily Rejectors particularly cite Too many
  steps to get what I needed
• Daily Indifferent cite Not that easy to find
  exactly what I wanted and Pages and layout too
  crowded

26
Source of Entry to Carlton.com

• The majority of visits (92) are from visitors
  who arrived by entering the URL or clicking
  bookmark for Carlton.com
• This is consistent with the high percentage of
  Heavy Users visiting the website
• The 4 of visits from visitors who arrived from
  a search engine are probably under-represented
  due to the large percentage of repeat visitors
  each month

27
Making a Reservation

• We saw earlier that 51 intended to make a
  reservation on the site today
• Reportedly 30 made a reservation on the site
  today. They are satisfied.
• 21 intended to make a reservation but did not
  do so!

28
Why Intended Reservation was Not Made

• We asked those who said they did not make a
  reservation (70), Why not?
• Half the 70 had no need right now
• 24 of non-bookers had solvable problems that
  were not solved by the site
• At least 15 of all visits intended to book and
  did not for solvable problems
• And, running into reservation problems also
  leads to becoming a Rejector

29
Future Reservation Plans If No Reservation Made

• Then we asked those who did not make a
  reservation what they planned to do
• Eighteen Percent (18) of the 70 opted for
  another medium
• Half the 70 have no need right now, thus 36
  of those with a reservation need opted for
  another medium
• Only 2 of every potential 3 is being captured
  by this low-cost channel
• Revenue is being lost to other channels, and
  perhaps completely lost

30
Business Travel

• The great majority of visits are by FBT

31
Business or Leisure Todays Visit

• However, more visits are for leisure or
  combined business/leisure than for business only

32
Demographics of Visitors

• Most visits are by male visitors

33
Income

• Most visits are by affluent visitors

34
Regions

• Regional differences are small, with somewhat
  higher Acceptor scores in the Southeast

35
Connect Speed

• The vast majority are Broadband vs. Dial-up
  users
• The site is equally satisfactory for Dial-up
  users as Broadband users

36
Quality Control on Research Instrument

• Weiss Linescale quality objective is to keep
  last two categories under 10, no matter the
  reason
• The interview is well received by Carlton.com
  visitors

37
Summary

• The majority (56) of visits are by very
  satisfied visitors, but close to 20 of visits
  have some issues
• More Elite Rewards members are most satisfied
  Basic and Non-Rewards members are less satisfied.
  Almost half the visits are by those who visit
  at least several times a week. However, even
  daily visitors indicate some problems areas
• Visitors do multiple activities (4.9) on the
  site each visit Check Rates and Rewards Points
  are the highest frequency activities. Other
  information activities tend to be somewhat
  lower-rated than reservation transaction
  activities.
• 30 reported making a reservation on the site
  today and 21 intended to make a reservation, but
  did not. Indications are revenue is being lost
  to other channels
• Detailed analytics are available showing
  reasons driving satisfaction and dissatisfaction,
  as well as activities done or intended by the
  satisfied and the less satisfied
• Linescale.com adds a Who, How and Why
  dimension to the Click and Transactional data
  currently available. This adds an in- depth
  understanding of total visitor flow and adds to
  business understanding of visitor and transactor
  behavior.

38
Next Steps

• Continue monthly tracking study with trend
  analysis, campaign analysis and further in-depth
  discriminant analysis. Develop further analytics
  marrying current web tracking and transaction
  data with Linescale data.
• Follow up with detailed analytics available
  within Linescale showing reasons driving
  satisfaction and dissatisfaction, as well as
  activities done or intended by the satisfied and
  the less satisfied
• Further targeted surveys based on these
  results
• Specific targeted studies can be done via same
  web invitation
• Focus on specific areas of Carlton.com
• Specific subsidiary websites
• Evaluate customer Acceptance and understanding
  whys for alternative paths, navigation and
  features prior to deployment via testing of
  screen shot flows or mini-sites
• Evaluate pre-post deployment Acceptance and
  understanding whys of performance of newly
  deployed features

39
Appendix Detailed Ratings
40
Activity Head-to-Head

• Example of Head to Head Paired Comparison
  results

41
Visitor Verbatim Comments on Why They Rated

• After rating, visitors commented on why they
  rated Carlton.com
• Comments are shown ranked from highest score
  (800 is perfect) to lowest
• Online at Linescale.com, each respondents
  summary data can be viewed along with the
  verbatim comment. The entire online interview is
  also viewable.