ICS481 Artificial Intelligence

1
ICS481 Artificial Intelligence

• Dr. Ken Cosh
• Lecture 7

2
Review

• Uncertain Reasoning
• Bayesian Networks
• Creation
• Inference
• Exact
• Approximate
• Sampling
• Alternative Approaches

3
This week

• Decision Making

4
Just Imagine

• Youve just won a tv gameshow 1,000,000
• But do you want to gamble? Toss a coin, if you
  win you get 3,000,000 you lose you get 0.
• Why?

5
Choices

• A 80 chance of 4,000
• B 100 chance of 3,000
• Why?

6
More Choices

• C 20 chance of 4,000
• D 25 chance of 3,000
• Why?

7
Quote

• In 1662, French Philosopher Arnauld said
• To judge what one must do to obtain a good or
  avoid an evil, it is necessary to consider not
  only the good and the evil in itself, but also
  the probability that it happens or does not
  happen and to view geometrically the proportion
  that all these things have together.
• More recently text move away from good and
  evil, and talk about utility.

8
Utility

• Suppose we can calculate the utility of any
  particular state given a utility function
• U(S)
• In reality this is often cumbersome, but for
  simplicity
• Should an agent perform action A, there are a set
  of different possible outcomes Resulti(A). Given
  the evidence (E), that the agent has about the
  environment probabilities for each Result can be
  assigned
• P(Resulti(A)Do(A), E)

9
Utility (2)

• We can then calculate the expected utility for
  performing that action given the evidence
• EU(AE) SP(Resulti(A)Do(A),E)U(Resulti(A))
• Maximum Expected Utility (MEU) states that a
  rational agent should choose the action which
  maximises the agents expected utility.

10
Maximum Expected Utility

• Great! Weve solved A.I.! All we need to do is
  calculate the action which is likely to return
  the maximum expected utility and set our agent
  loose!
• Sadly computations are often prohibitive.
• Knowing the initial state of the world requires
  perception
• Computing P(Resulti(A)Do(A),E) requires a
  complete causal model (NP-Hard reasoning in
  Bayesian Networks).
• Computing the Utility of each state
  (U(Resulti(A)) requires searching or planning as
  an agent cant assess the utility of a state
  until it knows where it can go from there.

11
Maximum Expected Utility

• Is it the only rational way?
• Why is maximising average utility so special?
• Why not minimise the worst possible loss?
• Couldnt an agent act rationally by expressing
  preferences between states without giving them
  numeric values?
• Perhaps a rational agent has a preference
  structure too complex to be captured by a simple
  number?
• Why should a suitable utility function exist at
  all?

12
Maximum Expected Utility

• To constrain the field of utility theory we will
  consider six axioms, known as the axioms of
  utility theory.
• The most obvious semantic constraints on
  preferences.
• Orderability
• Transitivity
• Continuity
• Substitutability
• Monotonicity
• Decomposability

13
But first

• Some notation
• A gt B (A is preferable to B)
• A B (The agent is indifferent between A
  and B)
• AgtB (The agent prefers A to B, or is
  indifferent between them)

14
Axiom 1

• Orderability
• Given any two states, a rational agent must
  either prefer one to the other or else rate the
  two as equally preferable. That is the agent
  cannot avoid deciding refusing to bet is like
  refusing to let time pass.
• (AgtB) ? (BgtA) ? (AB)

15
Axiom 2

• Transitivity
• Given any three states, if an agent prefers A to
  B, and prefers B to C, the agent must prefer A to
  C.
• (AgtB) ? (BgtC) ? (AgtC)

16
Axiom 3

• Continuity
• If some state B is between A and C in preference,
  then there is some probability p for which the
  rational agent will be indifferent between
  getting B for sure, and the lottery that yields A
  with probability p and C with probability 1-p.
• AgtBgtC ? ?p p,A 1-p, C B

17
Axiom 4

• Substitutability
• If an agent is indifferent between two lotteries,
  A and B, then the agent is indifferent between
  two more complex lotteries that are the same
  except that B is substituted for A in one of
  them. This holds regardless of the probabilities
  and the other outcome(s) in the lotteries.
• AB ? p, A 1-p, C p, B 1-p, C

18
Axiom 5

• Monotonicity
• Suppose there are two lotteries that have the
  same two outcomes, A and B. If an agent prefers
  A to B, then the agent must prefer the lottery
  that has a higher probability for A (and vice
  versa).
• AgtB ? (p q ? p, A 1-p, B gt q, A 1-q, B)

19
Axiom 6

• Decomposability
• Compound lotteries can be reduced to simpler ones
  using the laws of probability. This has been
  called the no fun in gambling rule because it
  says that two consecutive lotteries can be
  compressed into a single equivalent lottery.
• p, A 1-p, q, B 1-q, C p, A (1-p)q, B
  (1-p)(1-q), C

20
Axioms and Utility

• Clearly the Axioms weve discussed dont
  actually mention utility just preference.
• Fortunately preference is tightly linked to
  utility, as a rational agent should prefer
  options with higher utility.
• Preferences could be based on anything the agent
  likes for instance an agent might prefer prime
  numbers, or old cars.
• A utility function is more useful if preference
  is less arbitrary with money it is normal to
  want more money than less money.

21
Tossing the coin

• Youve just won a tv gameshow 1,000,000
• But do you want to gamble? Toss a coin, if you
  win you get 3,000,000 you lose you get 0.
• (0.53,000,000)(0.50) 1,500,000
• (11,000,000) 1,000,000
• Hence the Expected Monetary Value (EMV) of
  gambling is higher so why not gamble?

22
Utility and Money

• The expected utility of accepting and declining
  the gamble is not quite that straight forward
  The utility of winning your first million is very
  high, in comparison with winning a million if you
  are already very rich.
• EU (Accept) 0.5U(Sk) 0.5(Sk3,000,000)
• EU (Decline) U(Sk1,000,000)
• Research has shown that the utility of extra
  money is actually logarithmic rather than linear.
• If you already have 500,000,000, then gaining
  another 1,000,000 is worth almost the same as
  gaining 3,000,000.

23
Utility and Money

• Interestingly the logarithm curve is repeated
  below the 0 line. Someone with 10,000,000 debt
  might accept a gamble on a coin with 10,000,000
  gain or 20,000,000 loss.

24
Choices

• A 80 chance of 4,000 C 20 chance of 4,000
• B 100 chance of 3,000 D 25 chance of 3,000
• So what about these choices?
• A(0.84000)(0.20) 3200
• B30001 3000
• C(0.24000)(0.80)800
• D(0.253000)(0.750) 750
• (3200/3000)(800/750)
• Proportionally they are the same so why is B
  more appealing than A, and C more appealing than
  D. The answer lies in irrational regret.

25
Multiattribute Utility

• Money is a useful introduction to utility, but
  often preferences are made over several
  attributes
• For example when siting a new airport, we might
  consider cost, noise disruption, safety issues
  etc.
• For each option we can value each attribute to
  help us decide which is best.

26
Dominance

• An option is strictly dominated by another if it
  wins in all categories
• If airport location A is cheaper, quieter, and
  safer than B, then it has strict dominance.

In this deterministic example, B is strictly
dominant over A, while C and D are not.
B
C
A
D
27
Decision Network
Airport Site
Air Traffic
Deaths
Litigation
Noise
U
Construction
Cost
Ovals are Random Variables Rectangles are
Decision Nodes Diamonds are results of Utility
Function
28
Information

• Thus far we have assumed that all relevant
  information would be available to the agent to
  make their decision, however this is often not
  the case
• consider a doctor diagnosing a patient he cant
  possibly run every possible test.
• Hence it is worth trying to value information.

29
The value of Information

• Suppose an oil company hopes to buy one of n
  indistinguishable blocks of ocean drilling
  rights. Exactly 1 of the blocks contains oil
  worth C and the cost of buying each block is
  C/n.
• If a seismologist was selling information about
  certain blocks, say block 3, how much would that
  information be worth?

30
Oil

• Now there is a 1/n chance of oil in block 3!
• Then the company would buy block 3 and make the
  following profit
• C - C/n
• (n-1)C/n

31
No Oil

• There is a (n-1)/n chance of finding no oil. So
  the company would buy another block
• There is a 1/(n-1) chance of finding the oil in
  another block, and the expected profit is
• C/(n-1)-C/n
• C/n(n-1)

32
Overall

• Therefore the expected profit given the
  information is
• 1/n (n-1)C/n (n-1)/n C/n(n-1)
• C/n
• Ergo, the information is worth about as much as
  the block itself!

33
Assignment

• In honour of the honorable Jimmy
• Remember the Prisoners Dilemma from week 4 and
  the tough decision the prisoner had to make.
• PART 1 Consider our discussion about utility
  Suggest some other random variables which might
  affect our preference (such as honour amongst
  thieves) and discuss how it might change your
  answer.
• PART 2 What is the value of information?
  Firstly what is the value of knowing the
  punishments you might suffer, and secondly what
  is the value of knowing what your co-criminal has
  chosen?