Analysis Tools: Recursive Relations

1
Analysis Tools Recursive Relations

• Lecture 3 Algorithm Analysis
• Two Examples
• Nonrecursive Algorithms
• Sums
• Recursive Algorithms
• Difference Equations
• Empirical Analysis
• Performance Analysis

• Lecture 4 Analysis Tools Recursive Relations
• Difference Equations
• Existence and Uniqueness of Solutions
• Linear Difference Equations

Reference Introduction to Dynamic Systems,
D.G. Luenberger
2
Difference Equations

• Given a sequence of (real) numbers, y(k), a
  difference equation is an equation relating the
  value y(k), at a point k, to values at other
  points (usually neighboring points).

3
Difference Equations

• Some terms
• A function f(x) X ? R is linear in x if

• Is f(x)x2 linear? How about f(x) 7x2?

4
Difference Equations

• Some terms
• A function f(x) X ? R is linear in x if

• The order of a difference equation is the
• difference between the highest and lowest
• indices that appear in the equation.

5
Difference Equations

• Theorem a difference equation of order n is
  linear if and only if it can be written as
• where g(k) is arbitrary and ai(k) are functions
  independent of y for all i0,1,2,.

Proof Two directions to show. 1) If it has
that form, then linearity can be shown
directly. 2) If f y is linear in y, then
it will have a representation as multiplication
by an appropriate constant .
6
Difference Equations

• More terms
• The ai(k)s in these equations are called the
• coefficients of the linear equation.

• If these coefficients do not depend on k, the
• equation is said to have constant
  coefficients,
• or to be time-invariant.

• The function g(k) is called the forcing term,
  the
• driving term, the system input, or simply the
• right-hand side.

• A solution of the difference equation is a
  function
• y(k) that satisfies the equality relation.

7
Difference Equations

• Example 1 Consider

Verify y(k)1-1/k is a solution for k1,2,3,...
Verify y(k)1-a/k is a solution for any constant
a.
Can you find a solution?
8
Existence and Uniquenessof Solutions

• Revisit the first order, linear difference
  equation

Notice that for any finite collection of these
constraints there is exactly one more variable
than constraints.
We expect to have to assign a value to one of
these variables to specify a unique solution.
This extra value we assign is called the Initial
Condition.
9
Existence and Uniquenessof Solutions

• Revisit the first order, linear difference
  equation

In general, an nth order difference equation
will require the specification of n initial
conditions to obtain a unique solution (if it
exists).
10
Existence and Uniquenessof Solutions

• Existence and Uniqueness Theorem Let a
  difference equation of the form
• where f is an arbitrary real-valued function, be
  defined over a finite or infinite sequence of
  consecutive integer values of k (kk0, k01,
  k02,). The equation has one and only one
  solution corresponding to each arbitrary
  specification of the n initial values y(k0),
  y(k01),,y(k0n-1).

Proof Suppose the n initial values are
specified. Then the value of y(k0n) can be
uniquely determined simply by evaluating the
function f. The process then proceeds inductively
for each consecutive value of k.
11
Linear Difference Equations

• A linear difference equation, given by
• is said to be homogeneous if g(k)0 for all k in
  the set over which the equation is defined.

Note that 1) The difference of two solutions to
the nonhomogeneous equation must satisfy
the associated homogeneous equation, 2)
The sum of a solution of the nonhomogeneous
equation and a solution of the associated
homogeneous equation is a solution of the
nonhomogeneous equation.
12
Linear Difference Equations

• Theorem 1 Let y(k) be a given solution to the
  linear difference equation
• Then the collection of all solutions to this
  equation is the collection of all functions of
  the form y(k)y(k)z(k), where z(k) is a
  solution of the corresponding homogeneous
  equation.

13
Linear Difference Equations

• Theorem 2 If z1(k), z2(k), , zm(k) are
  solutions to the homogeneous equation
• then any linear combination of these m solutions
• where c1, c2, , cm are arbitrary constants, is
  also a solution of the homogeneous equation.

This theorem says linear combinations of
solutions are solutions, but it does not say all
solutions are found this way.
Proof Do it!
14
Linear Difference Equations

• Define a special set of solutions to the
  homogeneous equation corresponding to the n
  fundamental initial conditions
• Call this set of solutions z1,,zn the
  fundamental set of solutions.

...
15
Linear Difference Equations

• Theorem 3 If z(k) is any solution to the
  homogeneous equation,
• then z(k) can be expressed in terms of the n
  fundamental solutions, z1(k),,zn(k), of the form
• for some constants c1, c2, , cn.

This theorem says all solutions of the
homogeneous equation can be computed from the n
solutions generated by the fundamental initial
conditions.
16
Linear Difference Equations

• Theorem 3 If z(k) is any solution to the
  homogeneous equation,
• then z(k) can be expressed in terms of the n
  fundamental solutions, z1(k),,zn(k), of the form
• for some constants c1, c2, , cn.

17
Linear Difference Equations
Definition Given a finite set of functions
z1(k),,zm(k) defined for a set of integers,
say k0, 1, 2, , N, we say that these functions
are linearly independent if it is impossible to
find a relation of the form valid for all k0,
1, 2, , N, except by setting c1c2cn0.

• Theorem 4 Suppose z1(k),,zn(k) is a
  linearly independent set of solutions to the
  homogeneous equation
• Then any solution z(k) can be expressed as a
  linear combination
• for some constants c1, c2, , cn.

18
Linear Difference Equations

• Put it all together for a general method to
  solve a nonhomogeneous equation of the form
• which satisfies a given set of initial
  conditions.

• Find a set of n linearly independent solutions
  to
• the corresponding homogeneous equation,
• 2) Find a particular solution to the
  nonhomogeneous
• equation that does not necessarily satisfy the
  given
• initial conditions, and
• Modify the particular solution by adding a
  linear
• combination of the homogeneous solutions such
  that
• the given initial conditions are satisfied.