Survey of Decision Theory

1
Survey of Decision Theory

• in the Context of Top-k Querying

2
Motivation for Decision Theory

• Separate agents (not necessarily computers)
• All agents work together to make a decision that
  affects everyone
• Each agent has a utility for each possible
  outcome
• Each agent wants to maximize his own benefit
• Incentive to cheat (lie about your true
  preferences)

3
Applicability to Top-k

• Separate attributes
• Probably all computers, but
• Not necessarily the same computer
• Different scoring schemes
• Need to derive a single scoring function from all
  agents
• Utility?
• Certainly from the users perspective
• Accurate solution
• Cost of computing accurate solution?
• Could one query engine benefit from the final
  output?

4
Approaches to Decision Theory

• Voting
• Game Theory (Nash Equilibrium, Probability)
• Implies some sort of utility function to the user
  for each outcome (i.e. each possible top-k
  ordering?)
• Auctions
• Bargaining
• Both imply some sort of utility to each ranking
  agent for each outcome
• Contracts, coalitions, ?

5
Voting

• Combine preferences in a way that makes sense
  and is fair to all agents
• Not all agents will get their preferred outcome
• Example 2000 USA Presidential Election
• Normally we select Top-1 over all voters
  preferences
• Well extend it to Top-n for n candidates

6
Plurality Protocol

• Scores
• Bush 3
• Gore 2
• Nader 1
• Buchanan 1

• Or, remove winner and repeat for 2nd place
• Nader 3
• Gore 2
• Buchanan 2

7
Plurality Protocol Incentive to Cheat

• Scores
• Gore 4
• Bush 3
• Nader 0
• Buchanan 0

8
Plurality Protocol

• Advantages
• Easy to compute
• Easy to extend to top-k
• Disadvantages
• Incentive to cheat
• Winner may not rank highly in a majority

9
Binary Protocol

• Since more people preferred Gore to Bush,
  shouldnt Gore be ranked higher overall than
  Bush?
• Problem More people preferred Nader to Gore,
  shouldnt Nader be ranked higher?

10
Binary Protocol

• Ordering 1
• Bush gt Nader 4/7
• Bush gt Buchanan 6/7
• Gore gt Bush 4/7
• winner Gore

• Ordering 2
• Nader gt Gore 5/7
• Nader gt Buchanan 4/7
• Bush gt Nader 4/7
• winner Bush

11
Binary Protocol

• Advantages
• Winner must be preferred by the majority in some
  sense
• Disadvantages
• Can be non-deterministic, or at least has
  multiple equally valid outcomes
• Can lead to non-Pareto-efficient solution
• c gt d gt b gt a a gt c gt d gt b b gt a gt c gt d
• a wins over c, b wins over a, d wins over b,
  winner is d
• c is universally preferred over d

12
Arrows Impossibility Theorem

• All possible preferences must yield a valid
  overall ranking
• The outcome must be a total ordering
• Defined for every pair
• Asymmetric
• Transitive
• Pareto efficiency if everyone prefers a to b,
  then the outcome should rank a higher than b
• Independent of irrelevant alternatives
• No dictator outcome cannot follow a single
  voters preference regardless of the other voters

13
Borda Protocol

• Scores
• Bush 20
• Gore 17
• Nader 19
• Buchanan 14
• Outcome Bush gt Nader gt Gore gt Buchanan

14
Borda Protocol Irrelevant Alternatives

• Scores
• Bush 14
• Gore 13
• Nader 15
• Outcome Nader gt Bush gt Gore

15
Dictator Protocol
Always follow x1! Outcome Bush gt Buchanan gt
Nader gt Gore Or, pick a dictator at
random Outcome x7 (Buchanan gt Nader gt Gore gt
Bush) Not useful for Top-k. Or is it?
16
Dictator Protocol

• Advantages
• No incentive to cheat
• Satisfies all other criteria for Arrows theorem
• Cost to compute is cheap!!!!
• If most people prefer x, x has a good chance of
  ranking highly (Random dictator only)
• Disadvantages
• Doesnt take other rankings into account
• Winner may not be ranked highly by other agents

17
Plausibility Protocol
Plausibility of pairs Bush gt Buchanan 6 Nader
gt Gore 5 Bush gt Nader, Gore gt Bush, Nader gt
Buchanan, Gore gt Buchanan 4 Nader gt Bush,
etc 3 Gore gt Nader 2 Buchanan gt Bush 1
18
Plausibility Protocol

• Plausibility of pairs
• Bush gt Buchanan 6
• Nader gt Gore 5
• Bush gt Nader, Gore gt Bush, Nader gt Buchanan,
• Gore gt Buchanan 4
• Pick Bush gt Buchanan and Nader gt Gore.
• Pick the next preference as the one that
  eliminates the least of the other possibilities
• Bush gt Nader and Gore gt Bush eliminate each
  other, the other two eliminate nothing

19
Plausibility Protocol
Bush
Buchanan
Gore
Nader

• Bush gt Nader, Gore gt Bush 4
• 3
• Continue until total ordering is achieved.
  Break ties randomly.
• Two possible outcomes for this scenario

Bush
Gore
Buchanan
Nader
Bush
Gore
Buchanan
Nader
20
Plausibility Protocol

• Advantages
• Intuitive idea that the more a pairwise
  preference holds, the more likely it holds in the
  outcome
• Independent of irrelevant alternatives (?)
• Remove Buchanan, and the remaining preferences
  still have the same ranking
• Arrows Impossibility Theorem?
• Incentive to cheat?
• Disadvantages
• Costly to compute
• Can be non-deterministic

21
How Does This Fit In?

• Kießling paper Foundations of Preferences in
  Database Systems
• More about specifying preferences than decision
  theory
• All the papers that describe aggregation
  functions
• Scores on each item form a total ordering
• One query engine can dominate the final outcome
• No control over scoring functions
• One or more query engines has some vested
  interest in the final ranking? (e.g. Google gets
  some payola from Amazon.com)
• Perhaps we want to minimize the impact from
  artificially inflated rankings