Markov Random Fields and Gibbs Distributions

1
Markov Random Fields and Gibbs Distributions

• Qiang He
• School Of EE CS
• Oregon State University

2
Contents
1. Introduction
2. Nondirected graphs
3. Markov Random Fields
4. Gibbs Random Fields
5. Markov-Gibbs Equivalence
6. Inference tasks
7. Summary
3
1. Introduction
4

• Markov random fields (MRFs)

• A statistical theory for analyzing spatial
  contextual dependencies of physical phenomena.
• A Bayesian labeling problem
• A method to establish the probabilistic
  distributions of interacting labels
• Widely used in image processing and computer
  vision

5

• Properties of MRF

• Not ad hoc, can be solved based on sound
  mathematical principles (maximum a posterior
  probability, MAP)
• Incorporating prior contextual information
• Using local properties, which can be implemented
  in parallel

6

• An example image restoration using MRF

7
Image restoration process
Goals

• Restore degraded and noisy images
• Infer the true pixels from noisy ones

• Build the neighborhood systems and cliques
• Define the clique potentials for prior
  probability
• Derive the likelihood energy
• Compute the posterior energy
• Solve the MAP

8

• Definition for symbols

set of sites or nodes
neighbors
a nondirected graph
hidden true pixel
observed noisy pixel
9
2. Nondirected graphs
10

• Neighborhood Systems

• A neighborhood system for is defined as
• where is the set of sites neighboring .
  The neighboring
• relationship has the following properties
• a site is not neighboring to itself
• the neighboring relationship is mutual

11
Neighborhood Systems
12

• Cliques

A clique is defined as a subset of sites in
, where every pair of sites are neighbors of each
other. The collections of single- site,
double-site, and triple-site cliques are denoted
by , , and , A collection
of cliques is
13
Cliques
14
3. Markov Random Fields
15

• Basics

• Random field A family of rvs
• defined on the set
• Configuration a value assignment on a random
  field
• Probability
• --discrete case joint probability
• --continuous case joint PDF

16

• Markov random fields

• Positivity

• Markovianity

• Homogeneity probability independent of positions
  of sites

• Isotropy probability independent of orientations
  of sites

17

• Bayesian labeling problem

18
4. Gibbs Random Fields (GRFs)
19

• Gibbs distribution

• Partition function

Temperature
Energy function
Clique potentials
Special case Gaussian distribution
20
5. Markov-Gibbs Equivalence
21

• Proof MRFGRF

Conditional probability
Extended from clique potentials
Factor into two terms Containing i or not
Remove the term containing i
22

• MRF prior and Gibbs distribution

23

• Posterior MRF energy

Likelihood function
Likelihood energy
Posterior probability
Posterior energy
MAP solution
24
6. Inference tasks
25

• Goals

• Solve the Bayesian labeling problem, that is,
  find the maximum a posterior (MAP) configuration
  under the observation (simulated annealing
  process)
• Compute a marginal probability
• (Gibbs sampling)

• Parameter estimation

• Solve MRF prior probability through Gibbs
  distribution (since MRFGRF)
• Solve likelihood function by estimating the
  likelihood energy and the posterior energy
  coding method or least square error method
• Solve the MAP

26

• Look back at image restoration

• Build the neighborhood systems and cliques
• 4-neighborhood system and two-site cliques

• Define the prior clique potentials

27

• Compute the likelihood energy

• Compute the posterior energy

28
7. Summary
29

• The MRF modeling is to solve the Bayesian
  labeling problem, that is, find the maximum a
  posterior (MAP) configuration under the
  observation
• The MRF factors joint distribution into a product
  of clique potentials
• The MRF modeling provides a systematic approach
  in solving image processing and computer vision
  problems

30
Thank you very much!