NonNegative Linear Least Squares Problem

1
Non-Negative Linear Least Squares Problem

• Presented by Yi-Gang Tai

2
NNLS The Problem

3
An Example

• Find a solution for the equation Ax b where x
  0

x2
x1 1
2x1 x2 -2
x2 3
(x1 , x2) ?
x1
4
NNLS Algorithm

• Lawson Hanson (1974)
• Nine steps
• Set all elements of x to zero, and set all
  indices into set Z, set P to empty set.

5
NNLS Algorithm (contd)

• Main loop. Compute the gradient vector w from
  current value of x w AT(b Ax)

6
NNLS Algorithm (contd)

• If Z is empty or if all elements of w with
  indices in Z have values 0, we have a solution.
  Terminate.
• Z is not empty, and w has an element gt 0.
• Find the maximum element of w, which will be
  positive. Move its index from Z to P.

7
NNLS Algorithm (contd)

• Possible inner loop. Form Apos where the columns
  corresponding to indices in Z are replaced with
  columns of zeros. Solve the unconstrained linear
  least squares problem Aposz b. This will only
  determine the components of z corresponding to
  indices in P. Set the remaining elements of z to
  0.

8
NNLS Algorithm (contd)

• If all elements of z with indices in P are gt 0,
  this is an acceptable new trial solution. Set x
  z and continue with the main loop at step 2.

9
NNLS Algorithm (contd)

• Not all the effective elements in z gt 0, so we
  can only accept a fraction of z as the new trial
  solution. Find an index q such that xq/(xq - zq)
  is the minimum of all such expressions for
  negative elements in z. For this q, call the
  expression a.

10
NNLS Algorithm (contd)

• Form the linear sum x x a(z x)
• Move from P to Z all indices for which the
  corresponding element of x is zero. This will
  include q, and may include other elements as
  well. Continue the inner loop at step 5.

11
NNLS Algorithm (contd)

• Second iteration of the first example

12
Unconstrained Linear Least Squares Problem

• Solve Ax b
• QR decomposition

13
Computing QR by means of Householder
Transformation
14
The Example