Basic Probability Theory and Statistics

1
Basic Probability Theory and Statistics
Swipe
2
Basic Probability Theory and Statistics
These are some very fundamental terms/concepts
related to probability and statistics that often
come across any literature related to Machine
Learning and AI. Random Experiment Sample Space
Random Variables Probability Conditional
Probability Variance Probability
Distribution Joint Probability Distribution Condit
ional Probability Distribution (CPD) Factor
3
Random Experiment
A random experiment is a physical situation
whose outcome cannot be predicted until it is
observed.
4
Sample Space
A sample space, is a set of all possible outcomes
of a random experiment.
5
Random Variables
A random variable, is a variable whose possible
values are numerical outcomes of a random
experiment. There are two types of
random variables. Discrete Random Variable is
one which may take on only a countable number of
distinct values such as 0,1,2,3,4,.. Discrete
random variables are usually (but not
necessarily) counts. Continuous Random Variable
is one which takes an infinite number of
possible values. Continuous random variables are
usually measurements.
6
Probability
Probability is the measure of the likelihood that
an event will occur in a Random
Experiment. Probability is quantified as a
number between 0 and 1, where, loosely speaking,
0 indicates impossibility and 1 indicates
certainty. The higher the probability of an
event, the more likely it is that the event will
occur.
7
Conditional Probability

• Conditional Probability is a measure of the
  probability of an event given that (by
  assumption, presumption, assertion or evidence)
  another event has already occurred.
• If the event of interest is A and the event B is
  known or assumed to have occurred, the
  conditional probability of A given B, is usually
  written as P(AB).

8
Variance
The variance of a random variable X is a measure
of how concentrated the distribution of a random
variable X is around its mean.
9
Probability Distribution
Is a mathematical function that maps the all
possible outcomes of an random experiment with
its associated probability. It depends on the
Random Variable X , whether its discrete or
continues. Discrete Probability Distribution
The mathematical definition of a
discrete probability function, p(x), is a
function that satisfies the following
properties. This is referred as Probability Mass
Function. Continuous Probability Distribution
The mathematical definition of a continuous
probability function, f(x), is a function that
satisfies the following properties. This
is referred as Probability Density Function.
10
Joint Probability Distribution

• If X and Y are two random variables, the
  probability distribution that defines their
• simultaneous behaviour during outcomes of a
  random experiment is called a joint probability
  distribution.

11
Conditional Probability Distribution (CPD)
If Z is random variable who is dependent on other
variables X and Y, then the distribution of
P(ZX,Y) is called CPD of Z w.r.t X and Y. It
means for every possible combination of random
variables X, Y we represent a probability
distribution over Z. There are a number of
operations that one can perform over any
probability distribution to get interesting
results. Some of the important operations are
- Conditioning/Reduction Marginalisation
12
Conditioning/Reduction
If we have a probability distribution of n
random variables X1, X2 Xn and we make an
observation about k variables that they acquired
certain values a1, a2, , ak. It means we
already know their assignment. Then the rows in
the JD which are not consistent with the
observation is simply can removed and that leave
us with lesser number of rows. This operation is
known as Reduction.
13
Marginalisation
This operation takes a probability
distribution over a large set random variables
and produces a probability distribution over a
smaller subset of the variables. This operation
is known as marginalising a subset of random
variables. This operation is very useful when we
have large set of random variables as features
and we are interested in a smaller set of
variables, and how it affects output.
14
Topics for next Post
R-programming Data security Business
analytics Stay Tuned with