Least Squares Approximation: A Linear Algebra Technique

1
Least Squares Approximation A Linear Algebra
Technique

• By Larry Wong and James Sfregola

2
SooYou have a bunch of Data.
Attendence Big 4 Derby League Position per capita income adjusted population
39,968 1 0 11 35000 1582564
21,852 1 0 10 36725 137471
24,409 1 0 7 36725 371847
26,770 1 0 19 43600 852013
41,917 1 1 2 43600 1351187
37,162 1 0 6 36725 623160
24,510 1 1 16 43600 724121
43,958 1 1 3 36725 738873
40,699 1 0 14 36725 630043
75,595 1 1 1 36725 1194415

• What can we do with this?
• Use Least Squares Approximation (LSE) to estimate
  the relationship of the dependent variable and
  the independent variable(s).

3
What is Least Squares?

• Approximates solutions for inconsistent over
  determined systems
• What does inconsistent mean?
• Over determined, whats that?
• In other words LSE allows you to describe a model
  as being represented by the line of best fit.
• Where the best fit line, curve or polynomial is
  created from the approximations.

4
Least Squares its Actually Pretty Useful

• Least Squares Approximations are highly relevant
  in various fields
• Really, how so?
• Still dont believe us?
• Fine well give examplesbut not now.

5
So How Does it Estimate a Solution?

• Least Squares Approximation estimates a best fit
  solution for a system by
• Minimizing the magnitude error vector, e.
• Estimating the coefficients of the best fit
  equation

6
How Do We Ensure Positive Error?

• Least Squares
• Eliminates negative terms
• More susceptible to outlying data
• Least Absolute Difference
• Difficult to work with in linear algebra

7
(No Transcript)
8
The Least Squares Theorem

• For an over determined inconsistent system Axb
• This is easily solved using inverse and transpose
  multiplication
• the resulting vector x can be determined in the
  form x(AT A) -1 AT b
• where x will correspond to the coefficients in
  your linear or polynomial expression.

9
How to Approximate the Error Term?

• e b-Ax , where e is the error term
• Where e1 the distance from our data point to the
  best fit approximation given by the orthogonal
  projection

10
Application to Economics

• Econometrics utilizes Ordinary Least Squares
  Approximations (OLS)
• Similar to LSE except it follows 7 classical
  assumptions

Estimated Regression Equation
11
(No Transcript)
12
(No Transcript)
13
Bibliography

• Poole, David Linear Algebra A Modern
  Introduction. Canada Thomson Brooks/Cole, 2006.
• Studenmund, A.H. Using Econometrics A Practical
  Guide. New York Pearson Education, 2006.