Maximum likelihood and Bayesian Parameter Estimation

1
Maximum likelihood and Bayesian Parameter
Estimation
2
Overview

• Bayes formula
• Bayes decision rule
• Decide w1 if P(w1x)gtP(W2x) otherwise
  decide w2

3
Overview

• Parameter estimation
• ---Maximum likelihood estimation
• ---Bayesian estimation
• The parameters in MLE are fixed but unknown in
  BE the parameters are random variables having
  some known prior distribution.

4
Overview

• Maximum likelihood estimation

5
Bayesian estimation

• Prior density p(?) ? posterior density p(?D) ?
  class-conditional density p(xD)

6
Application Example

• Parameter estimation in Bayesian High-Resolution
  Image Reconstruction with Multisensors, IEEE
  transactions on Image Processing by Rafael
  Molina,2003.

7
Introduction

• Due to hardware and cost limitations, image
  systems often provide us with only multiple low
  resolution images, such as in remote sensing,
  surveillance and astronomy.
• Our goal is to reconstruct a high resolution
  image from multiple undersampled, shifted,
  degraded frames with sub-pixel displacement
  errors.

8
Problem Formulation

• Consider a sensor array with L1L2 sensors, where
  each sensor has N1N2 pixels and the size of each
  sensor element is T1T2.
• Our goal is to reconstruct an M1M2 high
  resolution image, where M1L1N1, M2L2N2, from
  L1L2 low resolution images.
• Here we consider the case L1L2L.

9
Problem Formulation
10
Problem Formulation

• In ideal case, the low resolution sensors are
  shifted with respect to each other by a value
  proportional to (T1/L)(T2/L).
• In practice, there can be small perturbations
  around these ideal locations, thus the horizontal
  and vertical displacements dl1,l2x and dl1,l2y of
  the l1,l2-th sensor with respect to the
  0,0-th reference sensor are given by

11
Problem Formulation
12
Image and degradation models

• Let f be the (M1M2)1 high resolution image and
  gl1,l2 be the (N1N2)1 observed low resolution
  image from l1,l2-th sensor.
• We want to construct f from gl1,l2 using Bayesian
  paradigm.

13
Image and degradation models

• The first step with this paradigm is the
  definition of a prior distribution over high
  resolution image f, p(fa), parameter a measures
  the smoothness of the high resolution image f.

14
Image and degradation models

• We also need to specify p(gl1,l2f, ßl1,l2), the
  probability distribution of the observed low
  resolution image gl1,l2 if f is the true high
  resolution image. vl1,l2 is modeled as
  independent white noise with variance ßl1,l2-1.

15
Image and degradation models
16
Bayesian Analysis

• Step 1 estimation of the parameters a and ß are
  selected as
• where

17
Bayesian Analysis

• Step 2 estimation of original high resolution
  image fa,ß is selected as the image satisfying

18
Bayesian Analysis
19
Bayesian Analysis
20
Experiment Result
21
My Conclusion

• These two parameter estimation techniques can be
  used in many applications
• ---the evaluation of employees job
  performance thus design an employee schedule
• ---the classifier of the scenery of SF into
  tree, street, building and ground etc
• --- Bayesian Methods for Cosmological Parameter
  Estimation from Cosmic Microwave Background
  Measurements.

22
My Conclusion

• They are classic parameter estimation techniques
  and their feature is using prior distribution
  information to determine unknown parameters in
  problems with known function form.
• In practice many problems will give us general
  knowledge about the situation, that is, we know
  the function form and prior distribution
  information, then we can use these techniques to
  solve the problems such as in image processing
  field.

23
references

• 1. Rafael Molina, Miguel Vega, Javier Abad and
  Aggelos K.Katsaggelos, Parameter Estimation in
  Bayesian High-Resolution Image Reconstruction
  With Multisensors, IEEE Trans. Image Processing,
  vol.12, No.12, December 2003.
• 2. Anze Slosar, Pedro Carreira, Kieran Cleary,
  etc, Cosmological parameter estimation and
  Bayesian model comparison using VSA data, Mon.
  Not. R. Astron. Soc. 000,1-6(2002).
• 3. Miguel Vega, Javier Mateos, Rafael Molina and
  Aggelos K.Katsaggelos, Bayesian Parameter
  Estimation in image construction from subsampled
  blurred observations, IEEE Trans.Image
  Processing,2003.
• 4. D.P. Morton and E. Popova, "A Bayesian
  stochastic programming approach to an employee
  scheduling problem," IIE Transactions on
  Operations Engineering 36 155-167 (2003).5.Nir
  Friedman and Yoram Singer, Efficient Bayesian
  Parameter Estimation in large discrete domains,
  NIPS, 1998.