Optimization

1
Optimization

• Lecture 8
• The Nature of Dynamic Optimization and Calculus
  of Variations

2
Example of a dynamic problem optimal investment
by the firm

• Suppose a firm uses a production function Q
  Q(K, L) with positive but diminishing marginal
  products. K is capital and L is labor
• The firm maximizes the net present value of the
  firm. This period profits are PQ(K, L) WL -
  mI, where P is the product price, W the nominal
  wage rate, m is the cost of capital, and I is
  gross investment
• I dK/dt ?K net investment plus depreciation

3
Investment Example (2)

• Suppose the firm has a discount rate r. We assume
  exponential discounting exp(-rt). Exponential
  discounting leads to a fast decay at the
  beginning of the period

exp(-rt)
t
0
4
Investment Example (3)

• So the net present value of the firm is
• N ?PQ(K,L)-WL-m(dK/dt ?K)exp(-rt)dt
• The problem with this objective function is that
  it contains both K and dK/dt
• We want to compute the optimal K and L
• L is not a big problem, but K is more complex

5
Example 2 Ramsey model

• Population ( labor force) N grows with n
• Output Y is produced using capital K and labor N
  Y F(K, N) C dK/dt C is consumption and
  dK/dt is investment. We assume that there is no
  depreciation
• We go to per capita terms y Y/N, k K/N, c
  C/N

6
Ramsey (2)

• There is a transformation problem. How to go from
  dK/dt to dk/dt?
• dk/dt d(K/N)/dt dK/dt.N-KdN/dt/N2 dK/dt/N
  - (K/N) dN/dt/N dK/dt/N - k.n
• So Y/N C/N dK/dt/N c dk/dt nk f(k)
• Inada conditions f(0) 0, f(0) ?, f(?) 0,
  and k0gt0
• We assume that consumers optimize utility
  U?0? u(ct) exp(-?t) dt

7
Ramsey (3)

• So maximize U?0?u(ct) exp(-?t) dt
• Subject to c dk/dt nk f(k) and kt, ct?0
• Again, this is a nasty problem
• The Ramsey model is an optimal savings model and
  currently a standard model of economic growth
• It is rather easy to solve the investment model
  that is what we do now the Ramsey-model needs a
  more advanced technique (namely Optimal Control)

8
Dynamic optimization

• All problems that have an intrinsic dynamic
  nature. The state of the world at time t is
  connected with time t1, t2, etc.
• This holds for all investment, growth,
  investment, even consumption decisions and models
• One can represent the problems also as
  multi-staging problems

9
Multi-stage decision making
State
7
8
2
Z
A
4
6
1
2
3
4
5
Stage
10
Continuous-variable version
State
B
A
Stage
0
11
General Dynamic Problem

• Max V ? f(x,u)dt
• Subject to dx/dt g(x,u)
• And some beginning and ending restrictions on x
  x(0) x0 and x(T) xT (see hereafter)
• x state variable, u control variable
• We are going to weight the restriction by a
  so-called co-state variable ?

12
Terminal conditions (we will return to these
later on)

• End-point condition
• xT xT
• xT is free
• xT ? xT

• Transversality
• None
• ?T 0
• ?T ? 0, xT-xT?T 0

13
Techniques

• Suppose there is no control variable in this
  case Calculus of Variations (Euler equations) is
  the appropriate technique
• Otherwise we can rely on (1) Lagrangean methods
  Optimal control using the Hamiltonian approach,
  (2) Dynamic Programming (Bellman equations)
  works for discrete problems

14
Calculus of Variations

• Classical approach to dynamic optimization
• Dates back to Newtons Principia (1687) and e.g.
  one of the Swiss Bernoulli brothers
• We transform the problem into
  max V ? f(t,x,dx/dt) dt
• Subject to a begin and end restrictions on x
  x(0) x0 and x(T) xT
• We need the functional to be integrable,
  continuous, and differentiable

15
Calculus of Variations intuition
x
xT
x(t) x(t) a h(t) pertubation
x0
x(t) optimal path
h(t) pertubation function
0
t
16
Interpretation

• We started seeing V as a function of x
• But now we can seen V as a function of a
• We know that the value of the problem is optimal
  for xx, so a0
• In this way dV/da0a0 is a necessary condition
  for the extremal

17
Calculus of Variations intuition

• For small a x(t) goes to x(t). Given x(t) and
  h(t) we need to have that dV/da 0
• Note that the pertubation function h(t) needs the
  properties h(0) 0 and h(T) 0
• Note that we do not know too much about a or
  h(t). Calculus of variations helps we formulate
  the Euler condition, which is a convenient way of
  expressing dV/da 0 at values of a close to 0
• In doing so we need to know how to differentiate
  an integral

18
Integration tricks

• Suppose we have I(x) ?ab F(t,x) dt. Leibnizs
  rule dI/dx ?ab Fx(t,x) dt
• Or denote I(a,b) ?ab F(t,x) dt.
• Then dI/da-F(a,x) and dI/db F(b,x)
• Integration by parts ?ab f(x)g(x) dx
  -?ab f(x)g(x)dx f(x)g(x) ab

19
Application to the pertubation integral

• Va ?0T f(t, xa, dxa/dt) dt
• dVa/da ?0T df/dx (dxa/da)df/d(dxa/dt)
  (d(dxa/dt)/da) dt ?0T fxh(t)fx h(t) dt
• Note that the last expression still contains the
  unknown h(t) and h(t). Now we use integration by
  parts on the last terms
• ?0T fx h(t) dt - ?0T h(t) d(fx)/dt dt

20
The Euler condition

• So we get ?0T fxh(t)fx h(t) dt
  ?0T h(t)fx- d(fx)/dt dt 0 for all
  h(t), so
• fx d(fx)/dt for all t in 0,T
• Special case 1 (no x) suppose we have
  f(t,dxa/dt). We will get d(fx)/dt constant
• Special case 2 (no t) f(x,dxa/dt). We get
  f dxa/dt Fx

21
More general Euler form

• fx d(fx)/dt is the standard Euler condition
• F is a function of three arguments t, x and x,
  so the total differential of d(fx)/dt consists
  of three terms
• d(fx)/dt ?fx/?t ?fx?x dx/dt ?fx?xdx/dt
  ftxfxx x(t) fxx x(t)
• So the Euler equation is fx ftxfxx x(t)
  fxx x(t) a second-order differential equation

22
Example Investment

• N ?PQ(K,L)-WL-m(dK/dt ?K)exp(-rt)dt
• df/dL PQL-Wexp(-rt) no dynamics for L
• df/dK PQK -m ? exp(-rt)
• df/d(dK/dt) -m exp(-rt)
• So d/dt df/d(dK/dt) r.m exp(-rt)
• The Euler-condition for K
  PQK-m ? exp(-rt) r.m.exp(-rt) or QK(?r)m/P

23
Example the Phillips-curve

• Loss function L (Yf -Y)2 ap2
• Phillips curve p - b(Yf -Y) pe
• Adaptive expectations dpe/dt j(p-pe)
• So we can write the loss function as
• L(pe,dpe/dt) (dpe/bj)2 a((dpe/dt/j)pe)2
• Minimize L(pe) ?0T L(pe,dpe/dt)exp(-rt) dt

24
Sidestep on dfx/dt

• dfx/dt can be expanded into
  f tx f x x x(t) fxx
  x(t)
• This is the total derivative. Note that f is a
  function of t, x, and x.
• So we get the Euler condition
  f txf x x x(t)fxx
  x(t)-fx0
• This is a second-order differential equation. We
  can use the begin and end conditions to pin down
  the integration constants

25
Application to the Phillips-curve

• Apply the previous slide to the inflation loss
  model (this is an exercise!) and find an equation
  like
• pe-rpe-Kpe0, where K (ab2j(rj))/(1ab2)
• So that the solution is of the form
  pe(t) A1 exp(r1t) A2 exp(r2t)

26
Easier example Eisner-Strotz Investment Model

• Maximize profits P(K) ?0? S(K)-C(K)
  exp(-rt)dt subject to K(0)K0
• Sales S(K) aK-bK2, a,bgt0
• Costs C(K) cK2 dK c,dgt0
• So F (aK-bK2 - cK2 dK)exp(-rt)
• We use the Euler condition
  f txf x x x(t)fxx x(t) fx
  0

27
Eisner-Strotz (2)

• FK (a-2bK)exp(-rt)
• FK -(2cKc)exp(-rt)
• FKK -2c exp(-rt), FKK0, FKK -2bexp(-rt),
  FtK r(2cKd)exp(-rt)
• Substituting into the Euler equation we get
• K - rK - (b/c) K (dr-a)/2c
• So the solution is of the form
• K(t) A1 exp(r1t) A2 exp(r2t) Kp

28
Summary

• Dynamic problems are abundant in economics all
  decisions depend on future expectations. Famous
  examples optimal savings, investment, and growth
  models
• Without control variables, calculus of variations
  in general and the Euler condition in particular
  are useful techniques
• With control variables we need to extend the
  model to optimal control problems