A Survey of Trust and Reputation Systems for Online Service Provision

1
A Survey of Trust and Reputation Systems for
Online Service Provision

• Audun Josang,
• Roslan Ismail, Colin Boyd

Presented by Matt Smith
2
Outline

• Introduction
• Background for Trust and Reputation Systems
• Security and Trust
• Collaborative Filtering and Collaborative
  Sanctioning
• Trust Classes
• Category of Trust Semantics
• Reputation Network Architectures
• Reputation Computation Engines
• Commercial and Live Reputation Systems
• Problems and Proposed Solutions
• Discussion and Conclusion

3
Introduction

• What risk exists in this online transaction?

Consumer Risk Has to pay for goods before
receiving them Cannot touch and try products
before buying
Service Provider Risk
Bob
Alice
Amazon
This asymmetric risk can be mitigated through
trust and reputation
4
Definition 1 (Reliability Trust)

• Trust is the subjective probability by which an
  individual, A, expects that another individual,
  B, performs a given action on which its welfare
  depends.

Gambetta (1998)
5
Definition 2 (Decision Trust)

• Trust is the extent to which one party is willing
  to depend on something or somebody in a given
  situation with a feeling of relative security,
  even though negative consequences are possible.

Inspired by McKnight Chervany (1996)
6
Definition 3 (Reputation)

• Reputation is what is generally said or believed
  about a persons or things character or standing
• I trust you because of your good reputation
• I trust you despite your bad reputation

Concise Oxford Dictionary
7
Principles of Reputation

• Reputation is one of the factors that trust is
  based on
• Reputation is someone elses story about me
• Reputation is based on identity
• Reputation exists in the context of community
• Reputation is a currency
• Reputation is narrative (evolves through time)
• Reputation is based on claims (verified or not),
  transactions, ratings, and endorsements
• Reputation is multi-level
• Multiple people holding the same opinion
  increases the weight of that opinion

Windley et al
8
Research Agenda

• Find adequate online substitutions for the
  traditional cues to trust and reputation systems
  that we are used to in the physical world, and
  identify new information elements (specific to a
  particular online application) which are suitable
  for deriving measures of trust and reputation.
• Take advantage of IT and the Internet to create
  efficient systems for collecting that
  information, and for deriving measures of trust
  and reputation, in order to support decision
  making and to improve the quality of online
  markets.

9
Properties of Reputation Systems

• 1. Entities must be long lived, so that with
  every interaction there is always an expectation
  of future interactions.
• 2. Ratings about current interactions are
  captured and distributed.
• 3. Ratings about past interactions must guide
  decisions about current interactions.

Resnick et al. (2000)
10
Trust Transitivity Principle
Reliability Trust Semantic Constraints
11
Security and Trust

• Hard Security
• Authentication
• Access Control
• Soft Security
• Social Control Mechanisms
• Trust and Reputation Systems

• Identity Trust
• A measure of the correctness of a claimed
  Identity over a communication channel
• Provision Trust
• The reliability of authenticated parties, or the
  quality of goods and services they provide

Rasmussen Jansson (1996)
12
Collaborative Filtering Sanctioning

• Collaborative Filtering
• Collect ratings
• Assumed subject to taste
• Optimistic world view
• Assumes all are trustworthy
• Rating things accurately improves your own
  recommendations

• Collaborative Sanctioning
• Collect ratings
• Assumed insensitive to taste
• Pessimistic world view
• Assumes some will have ulterior motives
• Sometimes rating things inaccurately may benefit
  you to achieve some goal

Reputation Systems
Recommendation Systems
13
Trust Classes
Grandison Sloman (2000)
14
Categories of Trust Semantics

• Interpretability of reputation ratings and scores

ex. Washing machine energy consumption ex.
Company fitness
15
Reputation Network Architectures

• Centralized
• Server / Clients

• Distributed
• Servents

ex. Peer-toPeer (P2P) Networks
ex. eBay, Epinions, Digg, etc.
16
Outline

• Introduction
• Background for Trust and Reputation Systems
• Security and Trust
• Collaborative Filtering and Collaborative
  Sanctioning
• Trust Classes
• Category of Trust Semantics
• Reputation Network Architectures
• Reputation Computation Engines
• Commercial and Live Reputation Systems
• Problems and Proposed Solutions
• Discussion and Conclusion

17
Reputation Computation Engines

• Simple Summation or Average of Ratings
• ex. Summation eBay, Average Epinions, Amazon.
• Bayesian Systems
• Discrete Trust Models
• Belief Models
• Fuzzy Models
• Flow Models
• Transitive iteration through looped or
  arbitrarily long chains
• ex. Google PageRank

18
Commercial and Live Systems

• eBay
• Expert Sites
• Product Review Sites
• Discussion Forums
• Google PageRank
• Supplier Reputation Systems
• Scientrometrics

19
eBay

• Buyers and Sellers can provide feedback
• Positive 1, Negative -1, or Neutral 0
• Can also leave comments
• 1 Smooth transaction, Thank you! (typical)
• -1 Buyers beware! (atypical)
• Resnick et al Findings (2002)
• Buyers provide ratings 51.7 of the time1
• Sellers provide ratings 60.6 of the time1
• 99 positive, neutral1
• Primitive reputation score
• Can be misleading 100 , 10 same 90 , 0
  
• Comprehensible
• Minor ratings problem Ballot stuffing limited
  by transactions
• Despite primitive nature and drawbacks the
  reputation system seems to have positive impact
  on eBay as a marketplace.

20
Expert Sites

• AllExperts site of self-proclaimed experts
• Rating on knowledge, clarity, timeliness, and
  politeness
• Score is numerical average of ratings
• AskMe closed to paying companies
• Advogato open-source programmer community
• Centralized, Flow model
• Others that you are familiar with?

21
Product Review Sites

• Epinions
• Reviews consist of
• 1 to 5 star ratings for a set of product aspects
• Written prose
• Users rate reviews
• Not Helpful, Somewhat Helpful, Helpful, or Very
  Helpful
• Simple Web of Trust
• Users can Block or Trust others
• BizRate
• Vulnerable to ballot stuffing
• Amazon
• Any registered user can comment on any product
• Was this review helpful to you? Yes or No
• Reports of many attacks ballot stuffing

22
Discussion Forums

• Slashdot news for nerds
• Kuro5in Technology and Culture
• Others (not in paper)
• Digg digg or bury stories
• SiteSays comment on any site as you browse
  users can rate any comment (except their own).

23
Google PageRank

• Ranks the best search results based on a pages
  reputation.
• Links from a web page can be seen as a positive
  rating
• Security by obscurity
• Trust transitivity to the extreme

24
Outline

• Introduction
• Background for Trust and Reputation Systems
• Security and Trust
• Collaborative Filtering and Collaborative
  Sanctioning
• Trust Classes
• Category of Trust Semantics
• Reputation Network Architectures
• Reputation Computation Engines
• Commercial and Live Reputation Systems
• Problems and Proposed Solutions
• Discussion and Conclusion

25
Problems

• Low Incentive for Providing Ratings
• Bias Toward Positive Rating
• Unfair Ratings
• Change of Identities
• Quality Variations Over Time
• Discrimination
• Ballot Box Stuffing

26
P1 Low Incentive for Providing Ratings

• Ratings are provided after a transaction has
  taken place
• Users withhold negative ratings because they are
  nice or fear retaliation
• Often a user does not benefit directly by
  providing a rating
• Nonetheless, many do provide ratings
• On eBay 60.7 of buyers and 51.7 of sellers do
  Resnick et al.
• Why? Perhaps, due to hopes of reciprocal ratings
• Why do people Digg stories?
• Miller et al. (2003) proposed a scheme for
  eliciting honest feedback based on financial
  rewards.
• Other incentives?
• Financial incentive (Cash Epinions, Discounts
  BizRate)
• Status or rank
• Community reciprocity?

27
P2 Bias Toward Positive Rating

• There is often a positive bias when ratings are
  provided
• On eBay only 0.6 of ratings by buyers and 1.6
  of ratings by sellers were negative Resnick et
  al.
• Other examples?
• Possible explanation
• Positive given in hope of receiving a positive
• Negative not given because of fear of retaliation
• Possible solution
• Anonymous reviews and/or ratings (e.g., Digg,
  SiteSays)
• Cryptographic scheme for anonymous ratings
  proposed by Ismail et al. (2003)

28
P3 Unfair Ratings

• Finding ways to avoid or reduce the influence of
  unfairly positive or unfairly negative ratings is
  a fundamental problem in reputation systems
• Categories of proposed solutions
• Endogenous Discounting of Unfair Ratings
• Exogenous Discounting of Unfair Ratings

29
Unfair Ratings Endogenous Discounting

• Description of Category
• Exclude or give low weight to presumed unfair
  ratings based on analyzing or comparing the
  rating values to themselves
• Assumption
• Unfair ratings can be recognized by statistical
  properties
• Proposed Solutions
• Statistical Analysis Dellarocas (2000) and
  Withby et al.
• Collaborative Filtering Chen Singh (2001)
• Other Endogenous Discounting Methods
• ?

30
Unfair Ratings Exogenous Discounting

• Description of Category
• Methods where the externally determined
  reputation of the rater is used to determine the
  weight given to ratings
• Assumption
• Raters with low reputation are likely to give
  unfair ratings and vice versa
• Proposed Solutions
• Bayesian Reputation Engines Buchegger Le
  Boudec (2003)
• P2P Network of Gnutella Cornelli et al. (2002)
• Trust Builder for rating subcontractors Ekstrom
  Bjornson (2002)
• Weighted Majority Algorithm variant Yu Singh
  (2003)
• Other Exogenous Discounting Methods
• Google PageRank
• ?

31
P4 Change of Identities

• Reputation systems generally assume
• Identities and pseudonyms are long-lived,
  allowing ratings about a particular party from
  the past to be related to the same party in the
  future.
• Changing identities is generally not in the best
  interest of the community Gambetta (1990)
• Proposed Solutions
• Penalize newcomers Zacharia et. al (1999)

32
P5 Quality Variations Over Time

• Description
• Economic Theory indicates that there is a balance
  between the cost of establishing a good
  reputation and the financial benefit of having a
  good reputation, leading to an equilibrium 37,
  62.
• Variations in the quality of service lead to
  variations in reputation
• Proposed Solutions
• Discounting of the past Huberman Wu (2003)
• Forgetting factor 32, aging factor 8, fading
  factor 7, longevity factor 31
• Discounting can be a function of time or of the
  frequency of transactions, or a combination of
  both 7

The numbered citations are consistent with the
paper being presented. Please see the paper for
full references
33
P6 Discrimination

• Description
• Discriminatory behavior can occur both when
  providing services and when providing ratings
• Examples and Possible Solutions
• A seller providing good quality to all buyers,
  except one (exogenous discounting methods are
  designed to solve this situation)
• A single rater giving fair ratings except when
  dealing with a specific partner (endogenous
  discounting methods are designed to solve this
  situation)

34
P7 Ballot Box Stuffing

• Description
• More than the legitimate number of ratings are
  provided.
• Solutions / Deterrents
• ratings can only be provided after a transaction
  (which costs money) eBay
• Only registered users can rate
• Other ideas?

35
Outline

• Introduction
• Background for Trust and Reputation Systems
• Security and Trust
• Collaborative Filtering and Collaborative
  Sanctioning
• Trust Classes
• Category of Trust Semantics
• Reputation Network Architectures
• Reputation Computation Engines
• Commercial and Live Reputation Systems
• Problems and Proposed Solutions
• Discussion and Conclusion

36
Basic Criteria for Judging the Quality and
Soundness of Reputation Computation Engines

• Accuracy for long-term performance
• Weighting toward current behavior
• Robustness against attacks
• Smoothness

Dingledine et al. (2000)
Josang et al. claim that criteria (1), (2), and
(4) are easily satisfied by most reputation
engines except for the most primitive (e.g., eBay)
37
Challenges Reiterated

• Unfair ratings
• Ballot stuffing
• Cheap pseudonyms
• Obtaining Ratings
• Little incentive

38
Reliability and Robustness

• Reliability of the current commercial systems is
  questionable
• Assuming that reputation systems give unreliable
  scores, why the are they used?
• Possible answers
• Even though it is not robust it might serve its
  purpose of providing incentive for good behavior
  if participants think it works Resnick et al.
• Even though it might not work well in the
  statistical normative sense, it may function
  successfully if it swiftly reacts against bad
  behavior ( stoning) and if it imposes costs for
  a participant to get established (initiation
  dues) Resnick et al.
• Do not need to be robust because their value lies
  elsewhere
• Serves as a social network to attract more people
• Positive bias may be desirable from a business
  perspective

39
Robustness

• Whenever robustness is crucial
• Measures should be taken to protect the stability
• Such as,
• Include routine manual control
• Keep the exact details of the computation
  algorithm and how the system is implemented
  confidential security by obscurity Epinions,
  Slashdot, Google
• They note, if ratings were objective it would be
  much simpler to achieve high robustness

40
No single solution

• There is no single solution that will be suitable
  in all contexts and applications
• Do you agree?

41
Conclusion

• Commercial
• Relatively simple schemes
• Academic
• Advanced features, but lack coherence
• Period of Pioneers
• We hope that the near future will bring
  consolidation around a set of sound and well
  recognized principles for building trust and
  reputation systems