L1 Regression

1

• L1 Regression
• LS (L2) is strongly affected by outliers
• If outliers are due to incorrect measurements,
  the inversion should minimize their effect on the
  estimated model.
• Effects of outliers in LS is shown by rapid
  fall-off of the tails of the Normal Distribution
• In contrast the Exponential Distribution has a
  longer tail, implying that the probability of
  realizing data far from the mean is higher. A few
  data points several ? from ltdgt is much more
  probable if drawn from an exponential rather than
  from a normal distribution. Therefore methods
  based on exponential distributions are able to
  handle outliers better than methods based on
  normal distributions. Such methods are said to be
  robust.

2

• L1 Regression
• min ? di -(Gm)i/?i min dw-Gwm1
• thus more robust to outliers because error is not
  squared
• Example repeating measurement m times
• 1 1 1T m d1 d2 dmT
• mL2 (GTG)-1GTd m-1 ? di
• f(m) d-Gm1 ? di-m
• Non-differentiable if mdi
• Convex, so local minimaglobal minima
• f(m) ? sgn(di-m), sgn(x)1 if xgt0, -1 if
  xlt0, 0 if x0
• 0 if half is , half is -
• ltdgtest median, where 1/2 of data is lt ltdgtest,
  1/2 gt ltdgtest

3

• L1 Regression
• Finding min dw-Gwm1 is not trivial.
  Several methods available, such as IRLS, solving
  a series of LS problems converging to a 1-norm
• rd-Gm
• f(m) d-Gm1 r1 ? ri
• non-differentiable if ri0. At other points
• ?f(m)/?mk - ? Gi,k sgn(ri) -? Gi,k ri/ri
• ?f(m) -GTRr -GTR(d-Gm)
• Ri,i1/ri
• ?f(m) -GTR(d-Gm) 0
• GTRGm GTRd R depends on m, nonlinear system
  (
• IRLS!