Chapter 8 Least Squares

1
Lecture 4

• Chapter 8 Least Squares
• Chapter 9 Methods of moments

2
Instructions
3
Least Squares (LS)

• NO probabilistic assumption about data!
• ONLY signal model assumed.
• NO optimality properties (more than LS)
• BUT, coincides with MLE, MVU if we are lucky

4
DC-level signal
5
Sinusoidal signal
6
The family of Least Squares

• Linear LS
• un-weighted
• weighted
• Order-recursive LS
• linear LS, but several model orders at once
• Sequential LS
• linear LS, on-line update, adaptive filter
• Constrained LS
• additional linear constraints on parameters
• Nonlinear LS
• make linear by transformation of parameter
• separate linear parameters
• numerical methods
• Geometrical interpretation possible

7
Linear least-squares
8
Order recursive least-squares 1
9
Order recursive least-squares 2
10
Sequential least-squares
11
Constrained least-squares
12
Nonlinear least-squares
13
Methods of moments

• NO optimality properties, but
• easy to implement
• usually consistent

14
MoM frequency estimator
15
MoM frequency estimation (cisoids)
16
Approximate ML as MoM
accurate, but high numerical complexity closed
form estimator
17
MoM frequency estimation
18
Performance analysis
19
Performance analysis 2

• Not the performance of the estimator, but the
  performance of a linearization of the estimator
• Denoted asymptotic variance
• Valid for
• high SNR / large number of data / both