Engineering Bargaining in Automated Negotiations

1
Engineering Bargaining in Automated Negotiations

• Francesco Di Giunta, Nicola Gatti
• Dipartimento di Elettronica e Informazione
• Politecnico di Milano

2
Introduction

• Game Theory Research Area
• Sequential Bargaining
• Engineering Research Area
• Computer Science Artificial Intelligence
• Aim
• Engineering bargaining to provide automation in
  bilateral transactions in electronic commerce
• Presentation structure
• Part One
• What is automated negotiation?
• Why bargaining in automated negotiations?
• Why game theory in automated negotiations?
• Part Two
• What bargaining model for automated negotiations?
• Results in the study of bargaining

3
Outline

• Introduction
• Part One
• Bargaining in Automated Negotiations
• Part Two
• Model of Bargaining
• Extensions and Results with Complete Information
• Extensions and Results with Incomplete Information

4
Automated Negotiations What?

• Electronic negotiations in which intelligent
  self-interested software agents negotiate with
  other agents on behalf of users for buying or
  selling services and goods Sandholm, 2000

5
Automated Negotiations Why?

• Increasing efficiency by saving resources
• Human work
• The agents act on behalf of the man
• Time
• The agents are faster than man
• Money
• The market is more free

6
An Example
_at_lleLunga - today 3x2
Bla, Bla
?
?
?
?
7
The Bargaining Problem

• A bargaining situation is characterized by the
  agents who have a common interest in cooperating,
  but who have conflicting interests concerning the
  particular way of doing so Napel, 2002
• Bargaining refers to the corresponding attempt to
  resolve a bargaining situation, i.e., to
  determine the particular form of cooperation and
  the corresponding payoffs for both

8
Bargaining History

• Ante Nash
• Edgeworth (1881), Zeuthen (1930), Hicks (1932),
  von Neumann and Morgenstern (1944)
• Nash
• Axiomatic approach (1950)
• Post Nash
• Strategic approach Rubinstein, 1982
• Incomplete information settings Rubinstein,
  1985
• Evolutionary approach Young, 1993

9
3 Questions in Automated Negotiation Study

• Why bargaining?
• Because it is the principal bilateral negotiation
  situation
• Why strategic bargaining?
• Because the agents interact according to a given
  protocol
• Why game theoretical strategic bargaining?
• Some discussion is needed

10
What if non Rational Agents?

• Bounded rational agents
• Agents have bounds that limit them in their
  maximisation process
• Ad-hoc rationality models Rubinstein, 1997
• Ad-hoc equilibrium notions Sandholm, 2001
• No rational agents
• Agents use rule of thumb learning from the
  observation of past negotiations Young, 1993
• Learning techniques for evolutionary game theory
  Binmore, Piccione, Samuelson, 1999

11
On Reasonability of the Rationality Models

• The question is what model of rationality among
  perfect rationality, bounded rationality, and no
  rationality is the most reasonable in automated
  negotiations?
• We analyse the assumptions that are needed to
  employ a rationality model
• Basically, a model of rationality is reasonable
  if its assumptions are reasonable

12
Perfect Rationality Assumptions

• Information
• Prior each agent has a prior about the opponent
• Common priors the priors is commonly known by
  agents
• Computability
• Agents as perfect maximisers agents can
  perfectly maximise their utility (i.e., they have
  not bounds)
• Common knowledge of 3 agents know that their
  opponent is a perfect maximiser

13
Assumption Reasonability

• Information
• Prior reasonable independently from the solution
  of the bargaining
• Common prior reasonable independently from the
  solution of the bargaining
• Computability
• Agents as perfect maximisers reasonable a
  posteriori the solution of the bargaining
• Common knowledge of 3 reasonable a posteriori
  the solution of the bargaining

14
Information Management
Bla, Bla
15
Assumption Reasonability

• Information
• Prior reasonable independently from the solution
  of the bargaining
• Common prior reasonable independently from the
  solution of the bargaining
• Computability
• Agents as perfect maximisers reasonable
  dependently on the solution of the bargaining
• Common knowledge of 3 reasonable dependently on
  the solution of the bargaining

16
Other Models Assumptions

• Bounded rationality
• Agents have bounds (informational and/or
  computational) that do not allow them to be
  perfect maximisers
• If perfect rationality is reasonable, then
  bounded rationality is not reasonable, given that
  the agents will try to maximise
• No rationality
• Agents do not accomplish any reasoning in
  negotiation, learning to play a Nash equilibrium
• If perfect rationality is reasonable, then no
  rationality is not reasonable, given that agents
  will know the optimal strategies and do not need
  to learn

17
Reasonability Summary

• If there exists a solution of the bargaining
  computationally tractable for the agents, then
  perfect rationality is the unique model of
  rationality reasonable in automated negotiations
• The reasonability problem is thus currently open!

18
Outline

• Introduction
• Part One
• Bargaining in Automated Negotiations
• Part Two
• Model of Bargaining
• Extensions and Results with Complete Information
• Extensions and Results with Incomplete Information

19
Strategic Models of Bargaining

• Classic Rubinstein, 1982
• Two players want to divide a pie of size 1
• Extensive form game where players alternately act
• Infinite horizon
• Complete information
• Revisited Computer Science Folk
• Two agents want to divide the surplus given by an
  economic transaction
• Each agent has a deadline after which he is not
  interested to the transaction any more
• The transaction can be valuated on several
  attributes
• Incomplete information

20
Classic Alternating-Offers

• Players
• Player function
• Actions
• Preferences

21
Equilibrium Notion and Solution

• Subgame Perfect Equilibrium Selten, 1972
• It defines the equilibrium strategies of each
  agent in each possible reachable subgame
• Typically, addressed using backward induction,
  but not in this case since the horizon is
  infinite
• Rubinstein Solution Rubinstein, 1982

22
A Graphical View
Infinite Horizon Construction
23
Outline

• Introduction
• Part One
• Bargaining in Automated Negotiations
• Part Two
• Model of Bargaining
• Extensions and Results with Complete Information
• Extensions and Results with Incomplete Information

24
Model Extensions

• Multiplicity of Issues
• The evaluation of each item takes into account
  several attributes xi
• Each offer is defined on all the attributes of
  the item, being a tuple x lt x1 , , xm gt
• Reservation Values (RVij)
• RVbj the maximum value of attribute j at which
  the agent b will buy the item
• RVsj the minimum value of attribute j at which
  the agent s will sell the item
• Deadlines (Ti) The time after which agent i has
  not convenience to negotiate any more
• Exit Option Agent can make exit at any time it
  plays

preferences
actions
25
Revisited Alternating-Offers

• Players
• Player function
• Actions
• Preferences

26
Complete Information Solution

• Backward Induction
• The game is not rigorously a finite horizon game
• However, no rational agent will play after his
  deadline
• Therefore, there is a point from which we can
  build backward induction construction
• We call it the deadline of the bargaining
• It is
• Solution Construction
• The deadline of the bargaining is determined
• From the deadline backward induction construction
  is employed to backward propagate the equilibrium
  offers

27
One Issue Backward Propagation
x
x
t
t-1
t-2
t-3
t
t-1
t-2
t-3
28
Backward Induction Construction
Infinite Horizon Construction
(RVs)?3bsb
(RVs)?2bsb
(RVs)?bsb
(RVs)?b
(RVs)?3bs
(RVs)?2bs
Finite Horizon Construction
(RVs)?bs
RVs
RVs
29
Equilibrium Strategies

• We call x(t) the offers found by backward
  induction for each time point t
• Equilibrium strategies are expressed in function
  of x(t)

30
Multi Issue Backward Propagation
How we can determine when issues are many?

• The agreement (x,t) is projected from subspace t
  to subspace t-1 according to Ui

• Among all the elements of Z, -i will chose the
  element that mazimises is utility

is Pareto efficient in subspace t-1
31
Multi Issue Backward Propagation Example x?b
t
t-1
RVb2
Us constant
x
x?b
RVb1
RVs2
RVs1
32
Backward Induction Construction
T
T-1
T-2
T-3
(RVs)?b
(RVs)?bs
RVs
RVs
(seller)
(buyer)
(seller)
(buyer)
33
Equilibrium Strategies

• We call x(t) the offers found with backward
  induction for each time point t
• Equilibrium strategies are expressed in function
  of x(t)

34
Computational Issues

• The maximisation problem in multiple issue
  backward propagation can be reduced to the
  fractional Knapsack problem
• The solving algorithm of complete information
  multiple issue bargaining has computational
  complexity
• When concave utility functions (for risk
  aversion) are employed computational complexity
  is due to the techniques for convex optimisation

35
Outline

• Introduction
• Part One
• Bargaining in Automated Negotiations
• Part Two
• Model of Bargaining
• Extensions and Results with Complete Information
• Extensions and Results with Incomplete Information

36
Summary

• Fudenberg and Tirole, 1991
• The theory of bargaining under incomplete
  information is currently more a series of
  examples than a coherent set of results.
• Classic results
• Uncertainty over one discount factor with two
  possible types Rubinstein, 1985
• Uncertainty over reservation prices with two
  possible types Chatterjee and Samuelson, 1992

37
Equilibrium of a Imperfect Information Extensive
Form Game

• Assessment (µ, ?)
• System of beliefs µ that defines the agents
  beliefs in each information set
• Equilibrium strategies ? that defines the agents
  action in each information set
• Equilibrium assessment
• Equilibrium strategies ? are sequentially
  rational given the system of beliefs µ
• System of beliefs are somehow consistent with
  equilibrium strategies µ

38
Notions of Equilibrium

• Perfect Bayesian Equilibrium (PBE) Fudenberg and
  Tirole, 1991
• Consistency is given by Bayes consistency on the
  equilibrium path, nothing can be said off
  equilibrium path, being Bayes rule not applicable
• Sequential Equilibrium (SE) Kreps and Wilson,
  1982
• Provide a criterion to analyse off-equilibrium-pat
  h consistency
• The consistency is given by the existence of a
  sequence of completely behavioural assessment
  that converges to the equilibrium assessment

39
How to Find Equilibrium Assessments?

• The circularity sequential rationality-consiste
  ncy introduces complications in the process of
  finding equilibria
• The main problem is how we can find Bayes
  consistent systems of beliefs

40
The Idea of Our Approach

• Call S the solution, and St a partial solution
  from t to the deadline of the bargaining
• We solve the problem of finding an equilibrium as
  a search problem
• We start backward from the deadline of the
  bargaining assigning
• At each level of the search tree it is assigned
  the assessment in one time point
• The successors of a node are given by all the
  possible systems of beliefs Bayes consistent with
  the construction made at the node
• We recall that by subgame perfection is St is not
  an equilibrium assessment in the pertinent
  subgame, then any solution St-k, that is an
  enlargement of St, is not an equilibrium
  assessment

41
How to Choose the Systems of Beliefs?

• From a node in which the assessment at time t has
  been assigned
• We consider all the kinds of equilibria in t
  given St1 (pooling and separating Fudenberg and
  Tirole, 1991)
• We consider one kind of equilibrium and we verify
  whether it gives rise to an equilibrium
  assessment
• We employ backward induction from St1 to time t
  to determine the presumed equilibrium offers ?
  according to the equilibrium kind we have
  considered
• We design a system of beliefs µt Bayes consistent
  with the optimal offers ? found at 3
• We compute the optimal offer ?t given the system
  of beliefs µt found at 4 and we verify whether ?
  ?t or not
• If ? ?t, then (µt, ?t) is an assessment
• Else, we go to step 2 considering a different
  kind of equilibrium

42
Rubinstein, 1985 with Deadlines
RVs
RVs
43
Rubinstein, 1985 with Deadlines
44
Remarks

• The equilibrium in pure strategies does not exist
  for some values of the parameters, whereas when
  the horizon is infinite equilibrium in pure
  strategies exists for all the values of the
  parameters
• The search can be computationally hard,
  exhibiting exponential complexity in the worst
  case

45
The Incomplete Information Settings We Studied

• One sided uncertainty over deadlines with
  finitely many types and pure strategies
• Non assured existence of equilibrium in pure
  strategies
• Computational complexity linear with the number
  of types
• One sided uncertainty over discount factors with
  finitely many types and pure strategies
• Non assured existence of equilibrium in pure
  strategies
• Computational complexity exponential with the
  number of types

46
What We Are Currently Studying

• Behavioural strategies in both the two previous
  incomplete information settings
• It assures the existence of the equilibrium for
  all the parameters
• It allows to reduce the computational complexity
  making it polynomial

47
Our Opinion

• It should be possible to find a solution of
  the bargaining problem, as formulated in the
  automated negotiation field, which is computable
  in polynomial time
• We are working to prove it!