Maximum Likelihood Estimation:

1
Maximum Likelihood Estimation Maximum
Likelihood is an ancient concept in estimation
theory. Suppose that e is a discrete random
process, and we know its probability density
function as a functional of a parameter ?, such
that we know P(e ?). Now we have n data
samples, given just as before ( y, u ), how do
we estimate ? ? The idea of Maximum Likelihood
Estimation is to maximize a Likelihood function
which is often defined as the joint probability
of ei.
2
Suppose ei is uncorrelated, the Likelihood
function L can be written as (the joint
probability of ei)
This means that the Likelihood function is the
product of data each samples pdf. Consider
using log Likelihood function Log L. Log
function is a monotonous function. This means
when L is maximum, so is Log L.
3
Instead of looking for , that maximizes L, We
now look for , that maximizes log L, the
result will be the same, but computation is
simpler!
4
If is Gaussian with zero mean, and
variance
Also consider the link between and data
observations is
5
(No Transcript)
6
By setting
We get
Which is simply equivalent to LS estimate.
A common fact Under Gaussian assumption, the
Least Squares estimates is equivalent to Maximum
Likelihood estimate.
7
Modelling Nonlinear AutoRegressive (NAR) Model by
Radial Basis Function (RBF) neural networks
e.g Gaussian Radial basis function
8
Radial Basis Function Neural Networks
9
Least squares (LS) can be readily used to
identify RBF networks.

• Some method to determine the centres (k-means
  clustering,
• or random selection from the data set), and
  given width s.
• 2. You know how to estimate ?.

is filled by