Convex Optimization

1
Convex Optimization

• Chapter 1 Introduction

2
What, Why and How

• What is convex optimization
• Why study convex optimization
• How to study convex optimization

3

• What is Convex Optimization?

4
Mathematical Optimization
Nonlinear Optimization
Convex Optimization
Least-squares
LP
5
Mathematical Optimization
6
Convex Optimization
7
Least-squares
8
Analytical Solution of Least-squares
9
Linear Programming (LP)
10

• Why Study Convex Optimization?

11
Solving Optimization Problems
Mathematical Optimization
Nonlinear Optimization
Convex Optimization
Least-squares
LP
12
Mathematical Optimization
Nonlinear Optimization
Convex Optimization
Least-squares
LP

• Analytical solution
• Good algorithms and software
• High accuracy and high reliability
• Time complexity

A mature technology!
13
Mathematical Optimization
Nonlinear Optimization
Convex Optimization
Least-squares
LP

• No analytical solution
• Algorithms and software
• Reliable and efficient
• Time complexity

Also a mature technology!
14
Mathematical Optimization
Nonlinear Optimization
Convex Optimization
Least-squares
LP

• No analytical solution
• Algorithms and software
• Reliable and efficient
• Time complexity (roughly)

Almost a mature technology!
15
Mathematical Optimization
Nonlinear Optimization
Convex Optimization
Least-squares
LP

• Sadly, no effective methods to solve
• Only approaches with some compromise
• Local optimization more art than technology
• Global optimization greatly compromised
  efficiency
• Help from convex optimization
• 1) Initialization 2) Heuristics 3) Bounds

Far from a technology! (something to avoid)
16
Why Study Convex Optimization

• If not, ?

there is little chance you can solve it.
-- Section 1.3.2, p8, Convex Optimization
17

• How to Study Convex Optimization?

18
Two Directions

• As potential users of convex optimization
• As researchers developing convex programming
  algorithms

19
Recognizing least-squares problems

• Straightforward verify
• the objective to be a quadratic function
• the quadratic form is positive semidefinite
• Standard techniques increase flexibility
• Weighted least-squares
• Regularized least-squares

20
Recognizing LP problems

• Example
• Sum of residuals approximation
• Chebyshev or minimax approximation

21
Recognizing Convex Optimization Problems
22
An Example
23
(No Transcript)
24
(No Transcript)
25
Adding linear constraints?????
26
Summary

• From the book, we expect to learn
• To recognize convex optimization problems
• To formulate convex optimization problems
• To (know what can) solve them!