The Value of Reputation on eBay: A Controlled Experiment

1
The Value of Reputation on eBay A Controlled
Experiment

• Andrew Berry
• 11/25/08

2
Internet Market

• No outside instrument of reputation
• Temptation for sellers to misrepresent products
  is great
• Temptation to sloth
• Example Ship slowly after receiving payment
• Buyers are forced to assume risk
• Should lower the price buyers are willing to pay

3
Internet Reputation Systems

• Necessary to substitute for traditional seller
  reputation mechanisms
• Inform buyers whether potential trading partners
  are trustworthy
• Deter opportunistic behavior
• Past actions affect future business
• Open record of transaction history

4
Internet and Reputation

• Information can be tallied costlessly on a
  continuous basis
• Written assessments are easily assembled
• Information can be costlessly transmitted across
  many customers

5
Prior Studies

• Observational studies of a set of items whose
  sellers had varying reputations
• Studies correlate reputations with auction
  outcomes
• Most studies found that buyers paid more to
  sellers with better reputations

6
Observational Studies

• Can only examine reputation in markets for
  standardized goods
• Plagued by omitted variable bias
• Discussion What factors could lead to OMVB?

7
Prior Work

• Shows that reputation affects
• Probability of a sale
• Price
• Probability that bidders enter an auction
• Number of bids in the auction
• Assessment of a sellers trustworthiness

8
Confounds with Observational Studies

• Private email communications
• Can influence buyer willingness to bid high
• Unobservable to researcher
• Layout aesthetics
• More experience may mean higher quality

9
Advantages to Field Experiments

• Automatically controls for confounds
• Ability to investigate reputation for
  non-standard goods with unavailable book value

10
eBay Reputation System

• To leave feedback a transaction must have
  occurred
• Buyer and seller can rate each other
• Opportunity for a one-line text comment
• Rated individuals can respond to comments that
  they feel are unfair

11
eBay Reputation System

• A buyer can click the net score in order to see a
  detailed breakdown
• Scroll to see individual comments
• Users may change identity by registering again
• No search mechanism to find negatives

12
General Page
13
Feedback Scores and Stars

• The Feedback score is the number in parentheses
  next to a members user ID
• Next to the Feedback score, you may also see a
  star  
• A Feedback score of at least 10 earns you a
  yellow star
• The higher the Feedback score, the more positive
  ratings a member has received
• As your Feedback score increases, your star will
  change color accordingly, all the way to a silver
  shooting star for a score above 1,000,000

14
(No Transcript)
15
Feedback Profile
A
C
B
D
16
Key Areas

• Positive Feedback Ratings
• The percentage of positive ratings left by
  members in the last 12 months.
• This is calculated by dividing the number of
  positive ratings by the total number of ratings
  (positive neutral negative).

17
Feedback Profile
A
C
B
D
18
Key Areas

• Recent Feedback Ratings
• The total number of positive, neutral, and
  negative Feedback ratings the member has received
  in the last 1, 6, and 12 months

19
Feedback Profile
A
C
B
D
20
Key Areas

• Detailed Seller Ratings
• provide more details about this members
  performance as a seller
• Five stars is the highest rating, and one star is
  the lowest
• These ratings do not count toward the overall
  Feedback score and they are anonymous
• Sellers cannot trace detailed seller ratings
  back to the buyer who left them

21
Feedback Profile
A
C
B
D
22
Key Areas

• All Feedback
• Provides feedback from all transactions
• Detailed user comments from transaction history

23
eBay Reputation Trends

• Half of the buyers provide positive feedback
• This positive feedback is similar to saying
  thank you in everyday discourse
• Sellers receive negative feedback only 1 of the
  time
• Buyers receive negative feedback only 2 of the
  time

24
Halftime

• Thought Questions
• 1. What do you think are biggest factors that
  account for so much positive feedback and so
  little negative feedback? Is the reputation
  system that good or is there something else at
  play?
• 2. Youve seen the eBay interface. Is there too
  much information to digest? What do buyers and
  sellers actually look at?

25
Experimental Setup

• 8 eBay identities
• STRONG
• Net score of 2000 with one negative feedback
• NEW
• 7 new eBay identities with no feedback
• Matched 200 items sold by STRONG with one of the
  new sellers

26
Experimental Setup

• Vintage postcards sold
• Asymmetry between seller and buyer about
  condition
• No established book value to guide buyers
• 12 week experiment
• 5 new sellers presented 20 lots each for sale
• 2 sellers presented 50 lots each

27
Experimental Setup

• To prevent customers from identifying the
  experiment
• Lots listed in a category that has thousands of
  lots for sale
• New sellers had slightly different format for
  listings
• Each half of each matched pair was listed at
  different times

28
Second Experiment

• Tested the effects of negative feedback
• 3 week experiment
• Purchased lots from three of the new sellers to
  give negative feedback
• Two categories of negative comments
• Item did not match description
• Item was in worse condition than listed

29
Second Experiment

• Negative feedback was displayed at the top of the
  comments page
• 35 more matched pair lots

30
Hypotheses

• Hypothesis 1
• Buyers will view an established seller as less
  risky and pay more
• Hypothesis 2
• New sellers with negative feedback will reap
  lower profits than those without negative
  feedback
• Thoughts on these hypotheses or the experimental
  setup?

31
Imperfect Observation

• Neither STRONG nor NEW sell
• Gives little information
• Either STRONG or NEW sells
• Provides a lower or upper bound on the ratio of a
  buyers willingness to pay
• Both STRONG and NEW sell
• Ideal situation

32
Slight Detour

• Censored Normal Regression Models
• Arise when the variable of interest is observable
  in certain conditions
• OLS is biased when the variable is unobservable
• Use these models when the independent variable is
  known, but the dependent variable is not
• Allows us to include data where either NEW or
  STRONG sold

33
Slight Detour

• Why dont we just use data where both sell?
• Reduce the sample size too much
• Truncation Bias
• New sellers sold fewer lots
• Observations of sold lots for NEW reflect more
  extreme points than for STRONG

34
Results

• Sign Test
• If STRONG sells but NEW does not, the sign is
  positive
• If both sell, the observed difference is used
• One sided sign test approaches significance
• Probability of sale was not independent of two
  sellers
• STRONG sold 63 of time
• NEW sellers sold 56 of the time

35
Results

• Censored normal estimation
• Parametric Estimate
• Used lots where either or both sellers sold
• Estimated mean difference is significant
• P .044
• Suggests buyers are willing to pay 8.1 more for
  lots sold by STRONG

36
Results

• Second experiment shows negatives in a brief
  reputation dont necessarily hurt revenues
• NEW sellers without negatives sold 16 of 35 lots
• NEW sellers with negatives sold 14 of 35 lots
• No significant differences
• Sellers without negatives often received lower
  prices when they did sell
• Favored sellers without negatives 9 times
• Favored sellers with negatives 11 times

37
Threats to Validity

• Experiment 1
• Differences in listing quality
• Repeat customers and private reputation
• Multiple purchases
• Experiment 2
• Small sample size
• Profile Design
• Timing of negative feedback

38
Discussion

• Validity of results?
• Is the percentage of negative feedback more
  valuable?
• Dewally and Edgerington (2006)
• Do buyers click through profiles or merely rely
  on overall score?
• How do we test if the market is over or
  underestimating reputation?

39
Discussion

• Given the results, what moral hazards does this
  pose for the structure of the eBay market?