COSC 4350 Artificial Intelligence

1
COSC 4350Artificial Intelligence

• Uncertain Reasoning Basics of Probability Theory
  (Part 1)
• Dr. Lappoon R. Tang

2
Overview

• What is uncertainty reasoning?
• Why is it an important topic in AI?
• Brief history of uncertainty reasoning
• Basics of probability theory

3
Readings

• R N Chapter 13
• Sec 13.1 (Focus on the motivation for uncertainty
  reasoning)
• Sec 13.2

4
What is uncertainty reasoning?

• Performing inference under uncertainty (i.e.
  incompleteness of information)
• Making rational decision that maximizes expected
  gain even though not all information necessary
  for making a perfect decision is available
• Example Stock markets, gambling,
• Need a way to characterize uncertainty (e.g.
  degree of uncertainty, likelihood, etc)
• Probability theory is our best choice of formalism

5
Why uncertainty reasoning?

• There are situations in real life that require
  reasoning and decision making based on uncertain
  evidence and incomplete information
• Classical logic only allows conclusions to be
  strictly true (or false)
• Problems
• Does not account for uncertainty
• Cannot weigh and combine evidences

6
Logical vs Uncertain Inference

• Logical
• Necessary
• Exact
• Logical formula
• (including True / False)
• Theorem proving

• Uncertain
• Not required
• Estimation
• Probability
• Probability calculations

• Features
• Complete Information
• Quality of conclusion
• Representation of conclusion
• Inference Mechanism

7
History of uncertain reasoning

• MYCIN (1975)
• One of the first expert systems ever created
  (applied on diagnosing bacterial infections),
  introduced rules with certainty factors. Uses
  some pretty ad hoc ways of combining evidence.
• Dempster-Shafer Theory (1976)
• Associates upper and lower bounds on beliefs and
  has clearer foundations in probability theory
• Bayesian Networks (1986)
• Use directed graphs to explicitly represent
  causal dependencies and use inference processes
  based directly on Bayesian probability theory
• Most popular form of probabilistic reasoning
  nowadays
• Combining logical and probabilistic reasoning
  (mid or late 80s - present)
• Explores ways to combining the strengths of the
  two paradigms into one, they come in various
  flavors
• Probabilistic relational models Friedman, et.
  al, 99
• Probabilistic logic programming Ng and
  Subrahmanian, 92
• A lot more

8
Probability Theory Basic Concepts

• True Story
• Many years ago, a student at Stanford
• University wrote the San Francisco Weather
• Bureau to ask what it meant by a 60
• chance of rain, did it mean it would rain
• over 60 of the area? Did it mean it would
• rain 60 of the time?.

9
Probability Theory Basic Concepts

• The Weather Bureau replied that it meant
• that there were ten men at the Weather
• Bureau and six of them thought that it was
• going to rain

10
Probability Theory Basic Concepts (contd)

• The lesson We need a way of defining the chance
  of seeing a certain outcome
• The sample space is the set of all possible
  outcomes (in an experiment)
• Example in tossing two coins, the set of all
  possible outcomes (H,H), (H,T), (T,H), (T,T)
• An event is a subset of the sample space
• Example in tossing two coins, the event at
  least one head appears (H,H), (H,T), (T,H)
• The sample space itself is also an event!
• An event is a particular collection of outcomes

11
Probability Theory Basic Concepts (contd)

• Let S be the set of all possible outcomes (i.e.
  the sample space). The unconditional or prior
  probability that an event A occurs P(A) (i.e. one
  of the outcomes of A happens) is given by
• Where A is the number of outcomes in A and S
  is the sample space

12
Probability Theory Basic Concepts (Examples)

• Example the set of all possible outcomes in
  tossing two fair coins (H,H), (H,T), (T,H),
  (T,T), the event A two coins turn up the same
  side consists of the set of outcomes (H,H),
  (T,T)
• So, p(A) A / S 2 / 4 0.5

13
Probability Theory Basic Concepts (Examples)

• Example the set of all possible outcomes in
  tossing two fair dices (1,1), (1,2), ,
  (6,6), the event A two dices sum to six
  consists of the set of outcomes (1,5), (2,4),
  (3,3), (4,2), (5,1) (there is no (0,6), so, it
  is not an outcome ?)
• So, p(A) A / S 5 / 36 0.139

14
Probability Theory Basic Concepts (Basic Laws)

• If S is a sample space, and A is the complement
  of A (i.e. A S A), then
• P(S) 1 (one of the outcomes of all possible
  outcomes has to occur)
• P(A) 1 P(A) (an outcome either occurs in an
  event or it does not in which case it occurs in
  the complement of the event)
• P( ) 0 (you cannot have no outcome)
• For any event A, 0 lt P(A) lt 1 (0 means event A
  can never happen, 1 means A will always happen)
• If A is a subset of B, then P(A) lt P(B) (the
  chance that an outcome occurs in an event with
  more outcomes is higher)