Lecture 1: Optimization Problem

1
Lecture 1 Optimization Problem

• Mainstream economics is founded on optimization
• The cornerstone of economic theory is rational
  utility maximization.
• In order to say something about how we expect
  economic man to act in this or that situation we
  need to be able to solve the relevant
  optimization problem.

2
The Structure of an Optimization Problem

• The basic elements of an optimization problem
  are
• the objective function, the available choices.
• The questions here is
• what do we want to accomplish?
• what can we do about it? and
• are there any restrictions on what we can do?

3

• Typically we are interested in questions such as
• Does a solution to the problem exist, i.e. is
  there a best course of action among the available
  alternatives?
• If a solution exists then what are its
  properties? Is it unique is it an interior
  solution or a corner solution etc?
• How does the solution change if the environment
  changes slightly, that is, what are the
  comparative statics properties?

4

• To be able to address these questions we need to
  look in depth on some mathematical properties of
  the objective function and the feasible set,
  i.e., the admissible set of actions or choices.
• Suppose we wish to maximize (or minimize) a
  function f by choosing some action x in the set
  of feasible actions X.
• Clearly, x maximizes f if f(x) f(x) for all
  x?X.

5

• What are the properties of the objective function
  and the feasible set that are of relevance for
  the existence and uniqueness of solutions to
  optimization problems.

6
The objective function

• Typical objective functions in economics are
  utility-, profit-, cost and welfare-functions.
• Two properties of particular interest concerning
  the objective function is
• continuity and
• concavity, or quasi-concavity.
• Continuity simply means that small changes in the
  arguments of the objective function should not
  lead to jumps in the value of the objective
  function. Specifically, we want to rule out ...
• Graph

7

• Concavity. A function f is said to be concave if,
  
• f(tx (1 t)x) tf(x) (1- t)f(x)
• Graph
• Quasi-concavity. A function f(x) is quasi concave
  if
• f(x) f(x) ? f(tx) (1- t)f(x)
  f(x) 0 t 1
• Graph

8
The feasible set

• We are interested in the following properties
• Non-emptiness If there are no admissible
  actions, e.g., due to mutually exclusive
  constraints, then a solution cannot exist.
• Closedness If the set is open then the optimal
  solution could lie on the border of the set which
  is not included in the set. For any given point
  in the set there is then always another slightly
  better point.
• Boundedness To ensure existence of a solution.
  For most economic problem this is a very natural
  restriction. A set that is closed and bounded is
  sometimes said to be compact.

9

• Convexity A set X is convex if x, x'?X and x
  kx (1-k)x'?X, k?0,1.
• It is strictly convex if a convex combination of
  boundary points, excluding the endpoints, lies in
  the interior of the set.
• Graph
• This affects whether solutions are local or
  global optima.

10
Existence

• Even for problems that are not mathematically
  well behaved there may well exist solutions in
  specific cases. It is, however, useful to know
  under what general conditions we can ensure that
  there exist a solution to an optimization
  problem.
• A continuous objective function together with a
  feasible set that satisfies the first three
  properties, i.e., non-emptiness and
  compactness,is by Wierstrass theorem sufficient
  to ensure that a solution exists.

11
Local and global optima

• Examples of local and global optima is contained
  in Graphs 2.8.
• (a) Non-convex feasible set.
• (b) Non-quasi-concave objective function.
• Graph
• A local maximum is always a global maximum if
• the objective function is quasi-concave, and
• the feasible set is convex.
• The idea is that if x is a local maximum and
  there exists a global maximum x' then a convex
  combination x is always as good as, and at the
  point x' better than, x since f is quasi
  concave. Hence, x cannot be a local optimum.

12
Uniqueness

• If the feasible set is convex and the objective
  function is non-constant and quasi-concave then a
  solution is unique if
• (a) the feasible set is strictly convex, or
• (b) the objective function is strictly
  quasi-concave, or
• (c) both
• Suppose x and x' are both optimal and let x be a
  convex combination.
• Since f is strictly quasi-concave this
  contradicts optimality. If the feasible set is
  strictly convex then x lies in the interior of
  the set.

13
Interior and boundary optima

• If the optimal point is an interior point, a so
  called Bliss point, then the solution is not
  affected by marginal changes in the constraints
  determining the feasible set.
• A relatively weak assumption ensuring that at
  least some constraint will bind is Local
  non-satiation.

14

• This means that it is always possible to find a
  small change in some variable that increases the
  value of the objective function.

15
The envelope theorem

• Consider a well behaved optimization problem
  where there exist a unique x that maximizes
  f(x,a) for any given value of the parameter a.
  The value of the optimal x typically depends on a
  and we can express this dependence as x(a).
• The maximal value of the objective function for a
  given a is then M(a) f(x(a),a). The effect on f
  of a change in a is,

16

• since the FOC for the choice of x requires that
• Thus the only effect of a change in a on the
  maximized objective function is the direct
  effect.

17
Comparative statics

• What is the effect of a parameter change on the
  optimal choice? This can be derived from the FOC.
  Differentiating the FOC with respect to a yields,
• Since the SOC requires the denominator in the
  last ratio to be negative the sign of dx(a)/da
  equal the sign of the cross derivative.

18
A few words on matrix algebra

• Consider the following linear equation system
• y 4 - 2/3 x
• y -1 x
• This can be rewritten as
• 2x 3 y 12
• -x y -1
• Or written on matrix form as follows,

19

• where the first factor is sometimes called a
  coefficient matrix.
• NOTE A convenient way of solving for x and y is
  to use Cramers rule which says that x is the
  ratio of two determinants based on the
  coefficient matrix,
• and

20

• In the first matrix the first column of the
  coefficient matrix is replaced by the column
  vector on the right hand side of the matrix
  equation.
• The determinant of a 2x2 matrix is the product of
  the north-west and south-east elements minus the
  product of the north-east and south-east
  elements.

21

• Applying this to our problem yields,
• To solve for y, we replace the second column by

22

• Note for larger matrixes we can expand along a
  row or column keeping in mind that the sign
  alternates depending on the position of the aij
  element and is given by (-1)ij. Below a 3x3
  example where we expand along the first column,

23
Unconstrained optimization with more than one
choice variable

• Consider the following maximization problem,
• At a peak the surface should be flat so that a
  very small move in any direction should leave us
  at the same altitude, i.e.,

24

• Thus the partial derivatives must all equal zero
  which means that the FOC is very similar to the
  one variable case. While the SOC looks different
  as compared to the one variable case the basic
  idea is the same.

25

• If we move away from the peak the terrain should
  slope downward in every direction. Thus,
• The reason that the condition is more complex is
  that it is not sufficient that 2nd derivatives
  with respect to each of the choice variables are
  negative.

26

• We also need to take the cross effects into
  account. Even if f11 lt 0 an increase in x1 may
  raise the marginal effects of other variables on
  f, e.g. f21 gt 0.
• The above expression can be restated in matrix
  form,

27

• It turns out that this condition holds if the
  matrix of 2nd derivatives, the Hessian, is
  negative semi-definite.
• This can be determined by checking the sign of
  determinant and the principal minors of the
  Hessian.

28

• Using a 2-Variable Example.
• The Hessian of f(x1, x2) is given by,
• This is negative definite if
• and

29

• Intuitively, the first expression tells us that
  fs degree of concavity wrt x1.
• Given that f11lt 0 the first term in the second
  expression can only be positive if f22lt 0.
  Moreover the product of these terms have to
  dominate the negative cross effect term (f12
  f21).

30

• Contour sets
• These are particularly useful for illustrating 2
  variable problems. (Examples indifference and
  iso-profit and iso-cost curves). The contour set
  of a function f of two variables x1 and x2 is
  defined as

31

• If the objective function is continuous then the
  contour set is also continuous. If f is
  differentiable then we can obtain the slope of
  the contour set can be obtained by
  differentiating f.
• If both f1 and f2 are positive the contour sets
  are downward sloping.

32

• The contour is said to be concave if
• Graph
• The definition of quasi concavity implies that
  quasi concave functions have concave contours.

33
Constrained optimization The Lagrange method.

• Many maximization/minimization problems are
  trivial without constraints.
• For instance,
• Utility maximization without a budget constraint
  yields infinite consumption.
• Cost minimization without a quantity target
  yields zero production.
• Consider the following problem

34

• Intuitively, we are looking for a tangency point
  between the boundary of the feasible set and the
  highest possible contour of the objective
  function.
• Note that g(x1,x2) b defines a contour and that
  g1dx1 g2dx2 0.
• Thus, the slope of the contour is,

35

• Similarly, the slope of the contour of the
  objective function is
• In an optimal point it must thus be true that (1)
• (2)

36

• We can express (1) as
• where is simply the value of the ratio at the
  optimal point.
• Now we can rewrite the optimum conditions (i) and
  (ii) as follows

37

• These equations can be solved for the
• variables

38

• Note that the above conditions are the very same
  conditions that result if we maximize the
  Lagrange function

39

• Interpretation of ?
• The marginal burden on the constraint if we
  increase one variable is measured by the first
  derivative wrt that variable. The constraint may
  be expressed in different units than the
  objective function.
• For example, a small change in consumption of a
  good has a monetary effect on the budget
  constraint but its effect on the objective
  function is measured in terms of utility.

40

• Also, if increasing one variable has a greater
  effect on the constraint it must have a
  correspondingly greater effect on the objective
  function.
• ? can be said to be the conversion factor that
  tells us how much effect we get per unit of the
  constraint.

41
The Envelope Theorem in a Lagrange Problem

• s.t.
• The Lagrange function is L

Let x be the optimal x and let v be defined by.
42

• The envelope theorem then implies that
• To see this we differentiate v totally
• where the terms in brackets equal 0 by the FOC.

43

• Example Consider the following maximization
  problem
• s.t.

44

• From the Langrage
• (1)
• (2)
• (3)

45

• From (1) and (2)
• From (3)
• Again, recall the maximized expression v and
  differentiate v totally

46

• The Envelope Theorem
• The envelop theorem says that the total effect on
  the optimized value of the objective function
  where a parameter changes ( and so, presumably,
  the whole problem must be reoptimized) can be
  deduced simply by taking the partial of the
  problems langragian with respect to the
  parameter and then evaluating that derivative at
  the original problems first order Kuhn-Turker
  conditions.

47

• The theorem applies regardless of the number of
  constraints, with the usual proviso that there be
  fewer constraints than choice variables. Note the
  theorem is not so obviously true.