Lecture 16' Convexity, Concavity

1
Lecture 16. Convexity, Concavity optimization

• Learning objectives. By the end of this lecture
  you should
• Know the link between convexity, concavity and
  optimization function.
• Know more about the properties of convex and
  concave functions.
• Introduction Reminder.

Concave function
Convex function
2
2. More properties.

• If f and g are both concave functions then fg is
  also concave
• Proof.
• f is concave so f(x?) ?f(x) (1-?)f(x)
• g is concave so g(x?) ?g(x) (1-?)g(x)
• So f(x?)g(x?) ?f(x) (1-?)f(x) ?g(x)
  (1-?)g(x)
• Or f(x?)g(x?) ?f(x) g(x) (1-?)f(x)
  g(x)
• If f and g are both convex functions then fg is
  also convex
• If f is concave then f is convex
• If f is convex then f is concave.
• If f is concave and g is convex then f-g is
  concave.
• Linear functions are both concave and convex.

3
Exercise.

• Draw a function that is convex
• Draw a function that is both convex and concave
• Draw a function that is neither convex nor
  concave.

4
3. The importance of concavity and
quasi-concavity..

• If f(x) is quasi-concave then any local maximum
  x is a global maximum.
• Suppose not i.e. suppose there was more than
  one local maximum as in the figure. Then the set
  S x f(x) y is not convex for some values
  of y.
• Moreover, if f is concave and differentiable and
  it attains its maximum at an interior point of
  the set upon which the function is defined, then
  the first order conditions are necessary and
  sufficient.
• In other words, if we solve the first order
  conditions and then can prove the function is
  concave then we know we have found a global
  maximum

f is not quasi-concave
y
5
4. Finding if a function is concave

• In general it is not straightforward to use the
  definition.
• When f is differentiable, then we can check using
  an implication of concavity

f(x)
6
4. Finding if a function is concave- x is a
single variable

• First recall or note Taylors theorem for x
  xdx or x-x dx
• E.g. if f(x) x2 then

7
4. Finding if a function is concave- x is a
single variable

• First recall or note Taylors theorem for x
  xdx or x-x dx
• E.g. if f(x) x2 then
• Here we have for a concave function
• So
• Or
• Cancel terms and we get
• Since dx2 must be nonnegative then concavity
  means
• i.e. the second derivative is not positive.
• Strict concavity means that the second derivative
  must be negative.

8
4. Finding if a function is concave- x is a vector

• Its a very similar story we replace the
  variable x by the vector x, and remember to get
  the order of multiplication right.
• So,
• We also need the vector of first partial
  derivatives of f and the matrix of second order
  partial derivatives.
• H is called the Hessian of the function f. It is
  a term you should remember.

9
4. Finding if a function is concave- x is a vector

• Our concavity condition is now
• The Taylors approximation
• So
• Or
• Cancel terms and we get
• In other words for any dx?0, dxHdx must not be
  positive.
• A matrix which produces this property is called a
  negative semi-definite matrix.
• negative because dx H dx cannot be positive.
• semi-definite because dxHdx can be zero.

10
5. Negative definite matrices

• The leading principal matrices of a square matrix
  are the principal matrices found by eliminating
• The last n-1 rows and columns written as D1
• The last n-2 rows and columns written as D2
• The original matrix - Dn
• The leading principal minors of a matrix are the
  determinants of these leading principal matrices.
• Example the leading principal matrices are then
  Dn A and (D1 1). The determinants are 5 and
  1.
• Example 2. Find D1 D2 and D3
• If a square matrix is negative definite then the
  determinants have the following pattern

11
5. Negative definite matrices

• The leading principal matrices of a square matrix
  are the principal matrices found by eliminating
• The last n-1 rows and columns written as D1
• The last n-2 rows and columns written as D2
• The original matrix - Dn
• The leading principal minors of a matrix are the
  determinants of these leading principal matrices.
• Example the leading principal matrices are then
  Dn A and (D1 1). The determinants are 5 and
  1.
• Example 2. Find D1 D2 and D3
• If a square matrix is negative definite then the
  determinants have the following pattern
• If a square matrix is negative semidefinite then
  the determinants have the pattern with not
  all zero.

12
6. Putting it all together

• So given a function f(x)
• Find the Hessian matrix of second order
  derivatives, H.
• From H find the leading principal matrices by
  eliminating
• The last n-1 rows and columns written as D1
• The last n-2 rows and columns written as D2
• The original matrix - Dn
• Compute the determinants of these leading
  principal matrices.
• f is concave if the determinants have the
  following pattern (with not all zero)
• Note that if one determinant has the wrong sign
  you dont need to check the others youve
  proved that the function is not concave.
• Reading the problems (and the course test) will
  require you to understand the conditions for a
  function to be convex. You need to study and
  learn those conditions.

13
6. Putting it all together - example

• Let f(x) -x1x22
• Find the Hessian matrix of second order
  derivatives, H.
• The first order partial derivatives are
• So H is,
• From H find the leading principal matrices by
  eliminating
• The last n-1 rows and columns written as D1
  (0)
• The last n-2 rows and columns written as D2 H
• Compute the determinants of these leading
  principal matrices.
• Det. D1 0
• Det. H -4x22 which is negative
• f is concave if the determinants have the
  following pattern

14
6. Example 2

• Maximize f(x) -x12-2 x22
• The first order conditions are
• Is this a maximum?
• H is,
• From H find the leading principal matrices by
  eliminating
• The last n-1 rows and columns written as D1
  (-2)
• The last n-2 rows and columns written as D2 H
• Compute the determinants of these leading
  principal matrices.
• Det. D1 -2
• Det. H 8-0 which is positive
• f is concave if the determinants have the
  following pattern

15
6. Example with three variables

• Maximize f(x) -x12-2 x22-x32
• The first order conditions are
• Is this a maximum?
• H is,
• From H find the leading principal matrices by
  eliminating
• The last n-1 rows and columns D1 (-2)
• The last n-2 rows and columns D2
• The last 0 rows and columns D3 H
• Compute the determinants of these leading
  principal matrices.
• Det. D1 -2,

16
7. Summing up two variable maximization

• Differentiate f(x) and solve the the first order
  conditions are
• Check concavity of f to see if the conditions
  represent a maximum.
• i.e. check if
• Then

17
8. Summing up 3 variable maximization

• Differentiate f(x) and solve the the first order
  conditions are
• Check concavity of f to see if the conditions
  represent a maximum.
• i.e. check if
• Then
• Then