Optimality in strategic games, CP nets and soft constraints

1
Optimality in strategic games, CP nets
and soft constraints
2
Main aim

• To compare the notion of optimality used in many
  formalisms
• To throw the basis for exploiting results in one
  field and reuse them in the other field
• Strategic games
• Agent interaction while pursuing their own
  interest (payoff function)
• CP nets
• Agents qualitative and conditional preferences
• Soft constraints
• Agents quantitative preferences

3
Outline

• Strategic games
• Relation between CP nets and games
• Relation between soft constraints and games

4
Parametrized Strategic games

• A set of players 1,.., n
• For each player i
• A set of strategies Si
• A strict total order gti over Si depending on s-i
  (a joint strategy of all players but player i)
  payoff function
• Example (prisoners dilemma) 2 players, 2
  strategies (ci, ni) for each player i

5
Pure Nash equilibria

• A strategy si is a best response for i to s-i if
  si i si for all si in Si
• A joint strategy s is a pure Nash equilibrium if
  each si is a best response to s-i
• Also for all i, for all si in Si, si i si
• No player has regrets on the strategy he chose
• But there could be better joint strategies if
  more than one player changed its strategy
• In the example, one Nash equilibria (NE) (N1,N2)

6
Pareto efficient joint strategies

• No other joint strategy is better or equal for
  all agents, and better for at least one
• Example
• (N1,N2) unique Nash equilibrium
• All other joint strategies are Pareto efficient
  (PE)

7
Dominance between strategies

• A strategy si is never a best response for i if
  it is not a best response to any joint strategy
  s-i
• In the example for each player i, Ci is never a
  best response

8
Elimination of dominated strategies

• G ?NBR G
• G subgame of G
• For all i, each si in Si-Si is never a best
  response for i in G
• Eliminate from the strategies of each players
  those that are never a best response

9
Nash equilibria and strategy elimination

• If G ?NBR G, then s Nash equilibrium of G iff
  Nash equilibrium of G
• In the example Ci is nbr, thus G has one row
  and one column, which is the unique Nash
  equilibrium 1,1

10
From CP-nets to games

• Given a CP-net N, we build the game g(N)
• Players features
• Strategies of player i domain of feature xi
• Payoff function of player i CP table for xi
• Given s-i, si gti si iff s-ipar(xi) si gti si
  in the cp table for variable i
• Thm opt(N) NE(g(N))

11
Example CP net
fishgtmeat
peaches gt strawberries
12
CP net ? Param. Strategic Game
Three players 1 main course, 2 wine, 3
fruit Two strategies for each player S1 meat,
fish S2red, white S3peaches, strawberries
fishgtmeat
Payoff functions For 1 main course fish gt meat,
always For 2 wine fish, -- ? white gt red meat,
-- ? red gt white For 3 fruit peaches gt
strawberries, always
peaches gt strawberries
13
Example optimals and Nash equilibria

• Unique optimal for CP-net (fish, white, peaches)
• Hard constraints fish, peaches, fish ? white,
  meat ? red
• For the game
• Meat is nbr for main course
• Strawberries is nbr for fruit
• Once meat is eliminated, red is nbr for wine
• Nash equilibrium fish, white, peaches

14
From games to CP-nets

• Given a game G, we build a CP-net n(G)
• Feature xi player i
• Domain of xi strategies for player i
• Parents of xi all the other features
• CP table of xi s-i si gt si if si gti si given
  s-i
• Thm. NE(G) opt(n(G))

15
Example

• Two features x1, x2
• D(x1)c1, n1
• D(x2)c2,n2
• x1 depends on x2
• x2c2 n1 gt c1
• x2n2 n1 gt c1
• x2 depends on x1
• X1c1 n2 gt c2
• X1n1 n2 gt c2

• Hard constraints
• x2c2 ? x1n1
• x2n2 ? x1n1
• x1c1 ? x2n2
• x1n1 ? x2n2
• Unique solution x1n1, x2n2

16
Reduced CP-nets

• If y is a parent of x, but the preference over
  the domain of x does not depend on y, then we can
  remove y from the parents of x ? eliminate rows
• From a CP net N to its reduced version r(N)

17
Example reduced CP-net

• Two features x1, x2
• D(x1)c1, n1
• D(x2)c2, n2
• x1 depends on x2
• x2c2 n1 gt c1
• x2n2 n1 gt c1
• x2 depends on x1
• X1c1 n2 gt c2
• X1n1 n2 gt c2

• Two features x1, x2
• D(x1)c1, n1
• D(x2)c2, n2
• x1 and x2 independent
• For x1 n1 gt c1
• For x2 n2 gt c2

18
CP-net techniques in games

• From game G to n(G)
• From n(G) to r(n(G))
• Hard constraints for r(n(G))
• Optimals of r(n(G)) Nash equilibria of G

19
Example

• Two features x1, x2
• D(x1)c1, n1
• D(x2)c2,n2
• x1 depends on x2
• x2c2 n1 gt c1
• x2n2 n1 gt c1
• x2 depends on x1
• X1c1 n2 gt c2
• X1n1 n2 gt c2

20
Example reduced CP-net

• Two features x1, x2
• D(x1)c1, n1
• D(x2)c2, n2
• x1 depends on x2
• x2c2 n1 gt c1
• x2n2 n1 gt c1
• x2 depends on x1
• X1c1 n2 gt c2
• X1n1 n2 gt c2

• Two features x1, x2
• D(x1)c1, n1
• D(x2)c2, n2
• x1 and x2 independent
• For x1 n1 gt c1
• For x2 n2 gt c2

21
Example reduced CP-net

• Two features x1, x2
• D(x1)c1, n1
• D(x2)c2, n2
• x1 and x2 independent
• For x1 n1 gt c1
• For x2 n2 gt c2

• Hard constraints
• x1n1
• x2n2
• Thus x1n1, x2n2 unique optimal solution of the
  CP-net and Nash equilibrium of the game

22
Games and acyclic CP-nets

• From game G to r(n(G))
• If r(n(G)) is acyclic, then G has one Nash
  equilibrium, and linear time to find it

23
Combination operator

• Extensive (always) for all a,b in A, a x b ? a,b
• Idempotent for all a in A, a x a a
• Ex. max, min, and
• Ex. of instances fuzzy, classical
• It is possible to apply soft constraint
  propagation
• Strictly monotonic for all a,b,c in A, a lt b ? a
  x c lt b x c
• Ex. sum, product
• Ex. of instances weighted
• It cannot be idempotent and strictly monotonic at
  the same time

24
Pareto efficient joint strategies

• No other joint strategy is better or equal for
  all agents, and better for at least one
• Example
• (N1,N2) unique Nash equilibrium
• All other joint strategies are Pareto efficient
  (PE)

25
From soft CSPs to games a local approach

• From a soft CSP P to a game L(P)
• Graphical games the payoff of each player may
  depend on the strategies of a subset of agents
  (its neighbours)
• Players one for each variable
• Strategies for a player i all values in domain
  of xi
• Neighbours variables in the same constraint
• Payoff function of player i for strategy s
  preference for assignment s in constraints
  involving xi

26
Optimality in SCSPs, NE in Games

• From Games to CSPs
• Full power of SCSPs no needed to model NE

Game G
CP-net n(G)
Greco et al.2005
CSP C(G)
Optimality Constraints of N(G)
Equivalent
27
Optimality in SCSPs, NE in Games

• From a SCSP P to a game L(P)
• Local Approach
• Players one for each variable
• Strategies for a player i all values in domain
  of xi
• Payoff of player i for joint strategy s
  preference for assignment s in constraints
  involving xi

28
Example 1 Fuzzy SCSP ? game
local
X
Y
Z

• Three players x,y,z
• Two strategies a,b
• Payoff functions
• For x px(aa-)0.4, px(ba-)0.3
• For y
• p(aaa) min(0.4,0.4) 0.4
• p(aba) min(0.1,0.1)0.1
• ...
• Two Nash equilibria aaa and bbb
• Optimal solutions only bbb

(a,a) ? 0.4 (a,b) ? 0.1 (b,a) ? 0.3 (b,b) ? 0.5
(a,a) ? 0.4 (a,b) ? 0.3 (b,a) ? 0.1 (b,b) ? 0.5
29
Example 2 Fuzzy SCSP ? game
local
X
Y
Z

• Three players x,y,z
• Two strategies a,b
• Payoff functions
• For x px(aa-)0.9, px(ba-)0.6
• For y
• p(aaa) min(0.9,0.1) 0.1
• p(aab) min(0.6,0.2)0.2
• ...
• Two Nash equilibria aab and bbb
• Optimal solutions only aab, abb, bab, bbb

(a,a) ? 0.9 (a,b) ? 0.6 (b,a) ? 0.6 (b,b) ? 0.9
(a,a) ? 0.1 (a,b) ? 0.2 (b,a) ? 0.1 (b,b) ? 0.2
30
Strictly monotonic combination

• In general, no relationship between optimal
  solutions of P and Nash equilibria of L(P)
• However, some relationship exist if combination
  is strictly monotonic
• Thm. Soft CSP P with strictly monotonic
  combination ? Opt(P) ? NE(L(P))

31
Classical CSPs ? games

• Classical constraints are combined via logical
  and (which is not strictly monotonic)
• However, if we consider consistent CSPs, the
  result holds
• Thm. consistent CSP ? Sol(P) ? NE(L(P))

32
Optimality in SCSPs, NE in Games

• Given an SCSP P, build a game GL(P)
• Global mapping
• Players variables
• Strategies domain values
• Payoff for player x for strategy s preference
  value for that assignment (by looking at all
  constraints)
• Note same payoff for all players
• Theorem Opt(P) ? NE(GL(P))
• Subset relation for all classes of SCSPs

33
Optimality in SCSPs, PE in Games

• From a game G to an SCSP L(G)
• Variables players (n)
• Domains strategies
• Semiring Cartesian product of n semirings
• For each variable xi, one constraint involving xi
  and its neighborhood
• pref(t) (d1,...,dn), where dj 1j for j ? i,
  and di F(pi(t))
• F is bijection from the payoffs to preferences in
  a c-semiring
• Thm. Game G ? opt(L(G)) PE(G)

34
Example
Semiring weighted x weighted
(c1,c2) ? (7,0) (c1,n2) ? (10,0) (n1,c2) ?
(6,0) (n1,n2) ? (9,0)
X1
x2
(c1,c2) ? (0,7) (c1,n2) ? (0,6) (n1,c2) ?
(0,10) (n1,n2) ? (0,9)

• Optimal solutions
• (c,c) with pref. (7,7)
• (n,c) with pref. (10,6)
• (c,n) with pref. (6, 10)

• Pareto efficient joint strategies all but (1,1)

35
Optimality in SCSPs, PE in Games

• From SCSPs to Games
• If we use the local mapping
• Opt(P) ? PE(L(P))
• If we use global mapping
• Opt(P) PE(L(P))

36
Summary CP-nets and NE games 1-1
N not reduced
g
r
g
N reduced
g(N)
r
n
n(g(N))
37
Summary SCSPs and Games

• Nash Equilibria

• Pareto Efficient

Game
SCSP
Game
CSP
? x st.m.
Game
?
Game
local
local
SCSP
SCSP
global
global
Game
?
Game

38
References

• CP-nets and Soft Constraints
• Carmel Domshlak, Steven David Prestwich,
  Francesca Rossi, Kristen Brent Venable, Toby
  Walsh Hard and soft constraints for reasoning
  about qualitative conditional preferences. J.
  Heuristics 12(4-5) 263-285 (2006)
• C. Boutilier, R. I. Brafman, Carmel Domshlak, H.
  H. Hoos, and D. Poole. Preference-based
  constraint optimization with CP-nets.
  Computational Intelligence, 20(2)137157, 2004
• Games, CP-nets and Soft Constraints
• Georg Gottlob, Gianluigi Greco, Francesco
  Scarcello Pure Nash Equilibria Hard and Easy
  Games. J. Artif. Intell. Res. (JAIR) 24 357-406
  (2005)
• Krzysztof R. Apt, Francesca Rossi, K. Brent
  Venable,Comparing the notions of optimality in
  CP-nets, strategic games, and soft constraints,
  to appear in Annals of Mathematics and
  Artificial Intelligence.