Why do we Need Statistical Model in the first place?

1
Why do we Need Statistical Model in the first
place?

• Any image processing algorithm has to work on a
  collection (class) of images instead of a single
  one
• Mathematical model gives us the abstraction of
  common properties of the images within the same
  class
• Model is our hypothesis and images are our
  observation data
• In physics, can Fma explain the relationship
  between force and acceleration? ? In image
  processing, can this model fit this class of
  images?

2
Introduction to Texture Synthesis

• Motivating applications
• Texture synthesis vs. image denoising
• Statistical image modeling revisited
• Modeling correlation/dependency
• Transform-domain texture synthesis
• Nonparametric texture synthesis
• Performance evaluation issue

3
Computer Graphics in SPORE
4
What is Image/Texture Model?
texture
speech
Analysis
Analysis
Pitch, LPC Residues
P(X) parametric /nonparametric
Synthesis
Synthesis
5
How do we Tell the Goodness of a Model?

• Synthesis (in statistical language, it is called
  sampling)

Does the generated sample (experimental result)
look like the data of our interests?
Computer simulation
Hypothesized model
Does the generated sequence (experimental result)
contain the same number of Heads and Tails?
Flip the coin
A fair coin?
6
Discrete Random Variables (taken from EE465)
Example III For a gray-scale image (L256), we
can use the notation p(rk), k 0,1,, L - 1, to
denote the histogram of an image with L possible
gray levels, rk, k 0,1,, L - 1, where p(rk) is
the probability of the kth gray level (random
event) occurring. The discrete random variables
in this case are gray levels.
Question What is wroning with viewing all pixels
as being generated from an independent
identically distributed (i.i.d.) random variable
7
To Understand the Problem

• Theoretically, if all pixels are indeed i.i.d.,
  then random permutation of pixels should produce
  another image of the same class (natural images)
• Experimentally, we can write a simple MATLAB
  function to implement and test the impact of
  random permutation

8
Permutated image with identical histogram to lena
9
Random Process

• Random process is the foundation for doing
  research in the field of communication and signal
  processing (that is why EE513 is the core
  requirement for qualified exam)
• Random processes is the vector generalization of
  (scalar) random variables

10
Correlation and Dependency (N2)
If the condition
holds, then the two random variables are said to
be uncorrelated. From our earlier discussion, we
know that if x and y are statistically
independent, then p(x, y) p(x)p(y), in which
case we write
Thus, we see that if two random variables are
statistically independent then they are also
uncorrelated. The converse of this statement is
not true in general.
11
Covariance of two Random Variables
The moment µ11
is called the covariance of x and y.
12
Recall How to Calculate E(XY)?

X

Y
Empirical solution
Note When YX, we are getting autocorrelation
13
Stationary Process
N
N
T
TK
space/time location
P(X1,,XN)P(XK1,,XKN) for any K,N (all
statistics is time invariant)
order of statistics
14
Gaussian Process
With mean vector m and covariance matrix C
For convenience, we often assume zero mean (if
it is nonzero mean, we can subtract the mean)
For Gaussian process, it is stationary as long
as its first and second order statistics are
time-invariant
The question is is the distribution of
observation data Gaussian or not?
15
The Curse of Dimensionality

• Even for a small-size image such as 64-by-64, we
  need to model it by a random process in
  4096-dimensional space (R4096) whose covariance
  matrix is sized by 4096-by-4096
• Curse of dimensionality was pointed out by E.
  Bellman in 1960s but even computing resource
  today cannot handle the brute-force search of
  nearest-neighbor search in relatively
  high-dimensional space.

16
Markovian Assumption
Andrei A. Markov 1856 - 1922
Pafnuty L. Chebyshev 1821 - 1894
Andrey N. Kolmogorov 1903 - 1987
17
A Simple Idea
The future is determined by the present but is
independent of the past
Note that stationarity and Markovianity are two
orthogonal perspectives of imposing
constraints to random processes
18
Markov Process
N-th order Markovian
N past samples
Parametric or non-parametric characterization
19
Autoregressive (AR) Model

• Parametric model (Linear Prediction)

z-transform
An infinite impulse response (IIR) filter
20
Example AR(1)
r(k)
Autocorrelation function
k
a0.9
21
Yule-Walker Equation
Covariance C
22
Wiener Filtering
In practice, we do not know autocorrelation
functions but only observation data X1,,XM
Approach 1 empirically estimate r(k) from X1,,XM
Approach 2 Formulate the minimization problem of
Exercise you can verify they end up with the
same results
23
Least-Square Estimation
M equations, N unknown variables
24
Least-Square Estimation (Cond)
rx
Rxx
If you write it out, it is exactly the empirical
way of estimating autocorrelation functions
now you have got the third approach
25
From 1D to 2D
6
2
3
4
4
2
3
Xm,n
1
5
Xm,n
1
5
6
7
8
Causal neighborhood
Noncausal neighborhood
Causality of neighborhood depends on
different applications (e.g., coding vs.
synthesis)
26
Experimental Justifications
AR model parameters
Analysis
original
random excitation
Synthesis
27
Failure Example (I)
Analysis and Synthesis
N8,M4096
Another way to look at it if X and Y are two
images of disks, will (XY)/2 produce another
disk image?
28
Failure Example (II)
Analysis and Synthesis
N8,M4096
Note that the failure reason of this example is
different from the last example (N is not large
enough)
29
Summary for AR Modeling

• We start from AR models because they are
  relatively simple and well understood (not just
  for images but also for speech coding, stock
  market prediction )
• AR model parameters are related to the
  second-order statistics by Yule-Walker equation
• AR model is equivalent to IIR filtering (linear
  prediction decorrelates the input signal)

30
Improvement over AR Model

• Doubly stochastic process
• In stationary Gaussian process, second-order
  statistics are time/spatial invariance
• In doubly stochastic process, second-order
  statistics (e.g., covariance) are modeled by
  another random process with hidden variables
• Windowing technique
• To estimate spatially varying statistics

31
Why do We need Windows?

• Nothing to do with Microsoft
• All images have finite dimensions they can be
  viewed as the windowed version of natural
  scenes
• Any empirical estimation of statistical
  attributes (e.g., mean, variance) is based on the
  assumption that all N samples observe the same
  distribution
• However, how do we know this assumption is
  satisfied?

32
1D Rectangular Window
X(n)
n
W(2T1)
33
2D Rectangular Window
Loosely speaking, parameter estimation from a
localized window is a compromised solution to
handle spatially varying statistics
W(2T1)
Such idea is common to other types of
non-stationary signals too (e.g., short-time
speech processing)
W(2T1)
34
Example
As window slides though the image, we will
observe that AR model parameters vary from
location to location
A
B
Q AR coefficients at B and C differ from those
at A but for different reasons, Why?
C
35
What is Next?

• Apply linear transformations
• A detour of wavelet transforms
• Wavelet-space statistical models
• Application into texture synthesis
• From parametric to nonparametric
• Patch-based nonparametric models
• Texture synthesis examples
• Application into image inpainting