Area: Reputation Systems

1
Area Reputation Systems
Gayatri Swamynathan CS290F, 10/21/04
2
The world today

• Steady growth of self-organizing and distributed
  systems (e.g. KaZaA)
• Interaction among complete strangers, sharing
  information, buying/selling goods or services
• Risky yet profitable, hence popular

3
Understanding Cooperation, Trust and Reputation
(Historical perspective and relevance today)

• Cooperation
• the online world involves mutually distrusting
  parties
• pseudonyms change, easy to deceive,
• scale, millions of users
• How should our systems increase levels of
  cooperation?

4
Understanding Cooperation, Trust and Reputation
(Historical perspective and relevance today)

• Trust
• sellers customers must trust that sellers will
  provide the services they advertise, do not
  disclose private information
• buyers able to pay for goods or services, is of
  legal age to access some services, is purchasing
  on behalf of an organization
• Is trust transitive?

5
Understanding Cooperation, Trust and Reputation
(Historical perspective and relevance today)

• Reputation
• Reputation is a characteristic or attribute
  ascribed to a person
• Rating is a fundamental way of adding value to an
  entity
• Conventional offline markets are driven by
  personal or corporate reputations
• What about online markets? These need to build in
  specific reputation-based mechanisms
• Amazon, EBay, KaZaA

6
What is a Reputation System?

• An online reputation system or a trust system is
  one that aggregates, processes and disseminates
  feedback about participants' behavior history.
• 1. provide information that allows buyers to
  distinguish between trustworthy and
  non-trustworthy sellers
• 2. encourage sellers to be trustworthy
• 3. discourage participation from those who
  aren't.
• In effect, Increase Cooperation!!

7
KaZaA

• In Kazaa's reputation management system, each
  peer has a Participation Level that is based on
  the quality and amount of files that it shares.
• meant to reward peers who share many Integrity
  Rated files by providing those peers with
  increased bandwidth
• major issue
• does not punish those who do not demonstrate good
  behavior
• system also does not provide a mechanism by which
  peers can rate each other's files based on their
  quality, relying on peers to rate their own files
  instead.
• Self rating malicious peers

8
Challenges

• Anonymity Vs Accountability
• Scale
• Free Riders
• Non Participation in Feedback Systems
• Random Feedback to gain incentives
• Collusions (false positives, false negatives)
• Genuine (honest) newcomers distrusted
• Dishonesty pays better! (in terms of money, time
  and effort spent)

9
How about "No Reputation"

• - "market for lemons" George Akerlof

10
So now, What is our goal ?

• we need a good solid model to collect, process
  and disseminate reputation-related information
• incentivize honest behavior in the system

11
Typical Online Scenarios

• Centralized Systems (run by corporate policies)
• buyer/seller environments like EBay monetary
  exchange
• P2P Distributed Systems (Decentralized)
• typically no money, free (pirated)
• is reputation really needed here?
• Grid
• MANETS

12
How do you build a good Reputation?

• provide good service
• provide good referrals
• build up your trust factor
• transitive trust

13
Motivation the situation today

• what we need is a robust and decentralized model
  of trust and reputation
• what we see is that there are several research
  groups building such models
• probabilistic, statistical, game-theoretic models
  (EPD)
• all attempt to determine an accurate estimation
  of a participants reliability towards a future
  interaction/transaction

14
Motivation Project Focus

• Compare the main P2P reputation models existing
  today
• The idea is to see under the same set of
  resources, which ones work the best
• Qualitative comparison (parameters, factors and
  metrics)
• Quantitative comparison - Simulations

15
Challenges

• Disparity in the models, based on Approaches,
  Assumptions, Parameters/Metrics, etc
• Incomplete simulation information
• Different simulation environments
• Approaches some models have users report only
  complaints in the system
• The task is now to bring them to one common
  ground. And only then will any comparison be
  fair.

16
Qualitative Evaluation METRICS

• How do we measure performance of any reputation
  system?
• Overhead in terms of data storage, computation,
  control messages, computation
• Detecting Malicious behavior
• Mean Time to detect
• False Positives/Negatives detection
• Goodput of the system (did the reputation system
  increase the overall levels of cooperation?)

17
Quantitative Evaluation ENVIRONMENTAL PARAMETERS

• participants
• semi-permanent identities
• registration process (with some higher central
  authority)
• no estimate of duration of each interaction
  between participants,
• long-term presence in the system
• participants are autonomous - and how they behave
  is completely up to their discretion.
• each participant can valuate the services that
  other participants provide independently and
  subjectively, without any control on correctness
  of its opinion.
• decentralized !

18
Quantitative Evaluation SIMULATIONS
Bootstrap History
Performance Metrics
Reputation Model
Participants
Data Storage
19
A Robust Reputation System for P2P and Mobile
Ad-hoc Networks (Sonya Buchegger, Jean-Yves Le
Boudec)

• copes with falsely disseminated reputation
  information
• learn from observation made by others, before
  learning by own experience

20
A Robust Reputation System for P2P and Mobile
Ad-hoc Networks APPROACH

• the reputation of a given node is a collection of
  ratings maintained by others about this node
  (DECENTRALIZED)
• node i maintains two ratings about every other
  node j that it cares about
• reputation rating opinion formed by node i about
  node j's behavior as an actor in the base system
  (Rij)
• trust rating node i's opinion about how honest
  node j is as an actor in the reputation system
  (Tij)
• - additionally, Fij first-hand information about
  node j

21
A Robust Reputation System for P2P and Mobile
Ad-hoc Networks APPROACH

• Periodically publish the first hand information
  to a subset of the population (not Rij or Tij)
• Will that information be used?
• Based on trustworthiness
• Similarity in Rating
• Ratings used to make decision about other nodes
  in the reputation-based P2P system.

22
A Robust Reputation System for P2P and Mobile
Ad-hoc Networks DETAILS

• normal/misbehaving use a tolerance threshold r
• trustworthy/non-trustworthy tolerance threshold t

23
A Robust Reputation System for P2P and Mobile
Ad-hoc Networks DETAILS

• How to update first hand information
• node i thinks that there is a parameter theta
  (probability) by which node j will misbehave.
• theta drawn according to a distribution based
  on more observations made
• standard approach same weight to each
  observation. Here, less weight to observations
  received in the past (reputation fading)

24
A Robust Reputation System for P2P and Mobile
Ad-hoc Networks DETAILS

• New observation made, say with s observed
  misbehaviors and f correct behaviors
• a a s b b f (based on this, plot the new
  distribution)
• To discount past behavior
• a ua s b ub f

25
A Robust Reputation System for P2P and Mobile
Ad-hoc Networks DETAILS

• How to update reputation rating
• 2 events cause this
• when first hand information is updated
• when a reputation rating published by some other
  node is accepted and copied
• if k is trustworthy make updates
• Rij Rij wFkj
• if k is not trustworthy - deviation test
  (basically subtract Fkj and Rij, greater than
  0..incompatible)

26
A Robust Reputation System for P2P and Mobile
Ad-hoc Networks PERFORMANCE
27
A Robust Reputation System for P2P and Mobile
Ad-hoc Networks ISSUES

• redemption periodic re-evaluation, and
  reputation fading
• liars not punished reduce their impact on public
  opinion
• why not punish?
• will discourage honest reporting of observed
  misbehavior
• testimonial accuracy is evaluated according to
  belief of the node, AND overall belief of the
  network built over time
• Intoxication does not work because of reputation
  fading reinforces a belief (whatever passes the
  deviation test)

28
A Robust Reputation System for P2P and Mobile
Ad-hoc Networks ISSUES

• how are liar strategies thwarted?
• swapping good and bad won't pass deviation test
• considerably worsen, or enhance reputation won't
  pass deviation
• stealthy approach lie only so much as to pass
  the deviation test. Impact is small as it, having
  passed the deviation test, only differs slightly
  from the reputation rating a node already has.
  the impact is further reduced by the fading and
  the limited frequency by which nodes consider
  second-hand information.

29
Conclusion Some future directions

• Post Transaction Buyer Analysis
• Collusions
• Reputation-history