Stochastic Modelling and Geostatistics

1
Lecture (5)
Introduction to Probability Theory and
Applications
2
Experiments

• Experiment a process that generates well-defined
  outcomes.

Experiment outcomes
Roll a dice 1,2,3,4,5,6
Sample space all possible outcomes. Sample
point any particle outcome.
For the Roll a dice example S1,2,3,4,5,6.
In the example 1.
3
Events

• An event is a collection of sample points (or, an
  event is a subset of a sample space)

(1) Rolling two dices (a) the sum of the
numbers that come up is odd, (b) the numbers that
come up are 3 and 6, etc. (2) Tossing two coins
at least one of them is head (it is a collection
of the following sample points (H,T), (T,H) and
(H,H)).
4
Coin Tossing
5

• Sometimes the experiment consists of several
  steps. In such a case, a tree diagram is handy.
  Consider the experiment of tossing two coins.
  S(H,H), (H,T), (T,H), (T,T)

Sample Point
Step 1 First Coin Toss
Step 2 Second Coin Toss
(H,H)
Head
Head
T
Tail
(H,T)
(T,H)
Tail
Head
Tail
(T,T)
6

• Complement of an event Given an event A, the
  complement of A is defined to be the event of all
  sample points that are not in A. The complement
  of A is denoted by Ac
• Venn Diagram

Sample space S
Ac
Event A
7
Rolling two dices (Example)
Event A at least one dice shows the number
1 Thus, event Ac none of the dices shows the
number 1
8
Union of two events

• the union of A and B is the event containing all
  sample points belonging to A or B or both. The
  union is denoted by

Sample space S
Event A
Event B
9
Rolling two dices (Example)
Event A at least one dice shows the number
1 Event B the sum of the numbers is at most 4
10
Intersection of two events

• Given two events A and B, the intersection of A
  and B is the event containing the sample points
  belonging to both A and B. The intersection is
  denoted by

Sample space S
Event A
Event B
11
Rolling two dices (Example)
event B
event A
Event A at least one dice shows the number
1 Event B the sum of the numbers is at most
4 The intersection (1,1), (1,2), (1,3), (2,1)
and (3,1)
12

• Mutually exclusive events two events are said to
  be mutually exclusive if the events has no sample
  points in common.

Sample space S
Event A
Event B
Events
Definitions
13

• Rolling two dices

event B
event A
Event A at least one dice shows the number
1 Event B the sum of the numbers is bigger than
10
14
Probability

• Probability is a number expressing the likelihood
  that a specific event will occur.
• Let Ei be a specific event (for example, the sum
  of the numbers on two dices is 2.)
• Let be the probability of this event.
• Axioms of probability

15
Assessing Probability
The probability of occurrence of a given event
can be assessed as the ratio of the number of
occurrences to the total number of possible
occurrences and non-occurrences of the event.
Occurrence of an event is called
Success Non-occurrence of the event is called
Failure The number of successes in N trials
n The relative frequency of successes n/N
16
Assessing Probability (Cont.)
17
Long Run Behavior of Coin Tossing
18
Probability ( Ex.)
Coin Sample Space H, T Do the following
experiments with one coin n5 n10 n20 n50 n10
0 Calculate p(H), p(T). Draw the convergence
curve between n and p(H), p(T).
19
Basic Probability Laws
where S is the set of all the sample points
Thus,
Example (tossing two coins) A is the event at
least one of them is head, and thus Ac is the
event none of the coins is head
Tossing two coins A is the event at least one
of them is head, and thus Ac is the event none
of the coins is head
and
and
Thus,
Thus,
20
Basic Probability Laws
Lets start with two mutually exclusive events.
In this case, and
Rolling two dices Event A at least one dice
shows the number 1, and event B the sum of the
numbers is bigger than 10
21
Basic Probability Laws (Example Solution)
Rolling two dices Event A at least one dice
shows the number 1, and event B the sum of the
numbers is bigger than 10

22
Basic Probability Laws
Why do we need to subtract
when the two events are not mutually exclusive?
Because otherwise we would double-count itboth
and include it.
Rolling two dices Event A at least one dice
shows the number 1, and event B the sum of the
numbers is at most 4
23
Basic Probability Laws (Example Solution)
Rolling two dices Event A at least one dice
shows the number 1, and event B the sum of the
numbers is at most 4
Dice 1
Dice 2
24
Conditional Probability
One of the most important concepts. It is denoted
as

which means the
probability of event A given the condition that
event B has occurred.
Rolling two dices Event A at least one dice
shows the number 1, and event B the sum of the
numbers is at most 4. What is the probability of
event A?
And what is the probability of event A if we know
that event B has occurred?

25
Conditional Probability (Cont.)
Event B
Event B with probability
26
General Multiplication Law
Multiplication Rule for Independent Events
Independent events are events in which the
occurrence of the events will not affect the
probability of the occurrence of any of the other
events.
27
The multiplication Rule for Independent Events
(Example)
Example Picking a color from a set of crayons,
then tossing a die. Separately, each of these
events is a simple event and the selection of a
color does not affect the tossing of a die.
If the set of crayons consists only of red,
yellow, and blue, the probability of picking red
is . The probability of tossing a die and
rolling a 5 is . Butthe probability of
picking red and rolling a 5 is given by
28
The multiplication Rule for Independent Events
(Example) Cont.
This can be illustrated using a tree diagram.
Since there are three choices for the color and
six choices for the die, there are eighteen
different results. Out of these, only one gives
a combination of red and 5. Therefore, the
probability of picking a red crayon and rolling a
5 is given by
29
The multiplication Rule for Independent Events
The general formula states
The multiplication rule for independent events
can be stated asThis rule can be extended for
more than two independent events
30
Multiplication Rule for Dependent Events
The occurrence of one event affects the
probability of the occurrence of other events.
An example of dependent events Picking a card
from a standard deck then picking another card
from the remaining cards in the deck. For
instance, what is the probability of picking two
kings from a standard deck of cards? The
probability of the first card being a king is
. However, the probability of the second
card depends on whether or not the first card was
a king.
31
Multiplication Rule for Dependent Events
(Example) Cont.
If the first card was a king then the probability
of the second card being a king is .
If the first card was not a king, the
probability of the second card being a king is
. Therefore, the selection of the first card
affects the probability of the second card.
32
Multiplication Rule for Dependent Events
(Example) Cont.
The multiplication rule that include dependent
events reads
33
Example 2
In a group of 25 people 16 of them are married
and 9 are single. What is the probability that
if two people are randomly selected from the
group, they are both married? If A represents
the first person chosen is married and B
represents the second person chosen is married
thenHere, is now the event of
picking another married person from the remaining
15 married persons. The probability for the
selection made in B is affected by the selection
in A.
34
Estimating Probability form Histogram
36
27
17.3
1.3
1.3
9
8
Probability that Q is 10,000 to 15, 000
17.3 Prob that Q lt 20,000 1.3 17.3 36
54.6
35
Estimating Probability form Cumulative Histogram
Continuous
F(x1) - F(x2)
Discrete
F(x1) P(x lt x1)
Prob that Q lt 20,000 1.3 17.3 36
54.6 Prob that Q 20,000, 80.6-54.6 27
36
Bivariate Distributions

• The bivariate (or joint) distribution is used
  when the relationship between two random
  variables is studied.
• The probability that X assumes the value x, and Y
  assumes the value y is denoted
• p(x,y) P(Xx and Y y)

37
Bivariate Distributions
38
Bivariate Distributions

• Example
• X and Y are two variables. Let X and Y denote
  the yearly runoff and rainfall (inches)
  respectively.
• The bivariate probability distribution is
  presented next.

39
Bivariate Distributions
0.42
p(x,y)
Example continued
X (in) Y (in) 0
5 10 0 .12 .42 .06 5 .21 .06 .03 10 .07 .02 .0
1
0.21
0.12
0.06
X
y0
0.06
0.03
0.07
0.02
y5
0.01
Y
y10
X0
X10
X5
40
Marginal Probabilities

• Example- continued
• Sum across rows and down columns

p(0,0)
p(0,5)
p(0,10)
The marginal probability P(X0)
41
Conditional Probability
Example - continued
42
Counting Techniques Permutations
Permutations It is an arbitrary ordering of a
number of different objects using all of
them. If we want to permute n different objects
in r arrangements The number of permutations of
n different objects is the number of different
arrangements in which r (rltn) of these objects
can be placed, with attention given to the order
of the items in each arrangement.
If nr,
The number of permutations of n different objects
of groups of which n_i are alike, is
43
Example 1
Example with objects a, b, and c how many
permutations can we do in 2 arrangements? ab,
ac, ba, bc, ca, cb First place we
have 3 choices. Second place we have two choice.
Altogether 3!/(3-2)!6 permutation of three
objects in two arrangements.
44
Example 2
Example with objects a, b, and c we can produce
six permutations abc, acb, bac, bca,
cab, cba First place we have 3
choices. Second place we two choices. Third place
we one choice. Altogether 321 3! 6
permutation of three objects. For n objects we
get n(n-1)(n-2)21n! permutations
45
Example 3
The number of permutations of n objects
consisting of groups of which n_i are alike, is
The number of permutations of letters in the word
STATISTICS 50400.
46
Counting Techniques Combinations
Combinations The number of combinations of n
different objects r at a time (ngtr) is the number
of different selection that can be made of r
objects out of n, without giving attention to the
order of arrangement within each selection.
Stirling formula
47
Example 1
Example with objects a, b, and c we can produce
six permutations abc, acb, bac, bca,
cab, cba 3!3216 But three
combinations abccbaacbbcabaccab 3!/3!.0!3
21/3211
48
Example 2
Example Country A 7 watersheds, and Country
B 4 Watersheds. Five watersheds have to be
selected for research 3 from A and 2 from B. How
many different ways this choice can be made?
Solution This is a problem of combinations
because the order of the watersheds is not
important. A n 7, r 3 B n4, r2