Online convex optimization Gradient descent without a gradient

1
Online convex optimizationGradient descent
without a gradient

• Abie Flaxman CMU
• Adam Tauman Kalai TTI
• Brendan McMahan CMU

2
Standard convex optimization

• Convex feasible set S ½ ltd
• Concave function f S ! lt

3
Steepest ascent

• Move in the direction of steepest ascent
• Compute f(x) (rf(x) in higher dimensions)
• Works for convex optimization
• (and many other problems)

x1
x2
x3
x4
4
Typical application

• Toyota produces certain numbers of cars per month
• Vector x 2 ltd (Corollas, Camrys, )
• Profit of Toyota is concave function of
  production vector
• Maximize total (eq. average) profit

PROBLEMS
5
Problem definition and results

• Sequence of unknown (concave) functions
• On tth month, choose production xt 2 S ½ ltd
• Receive profit ft(xt)
• Maximize avg. profit
• No assumption on distribution of ft

8 z, z 2 S z z D rf(z)
G
6
First try
Zinkevich 03 If we could only compute
gradients
f4(x4)
f3(x3)
f2(x2)
f4
PROFIT
f1(x1)
f3
f2
f1
x1
x2
x3
x4
CAMRYS
7
Idea 1-point gradient estimate
With probability ½, estimate f(x ?)/?
With probability ½, estimate f(x ?)/?
PROFIT
E estimate ¼ f(x)
x
x?
x-?
CAMRYS
8
Analysis
PROFIT
x?
x-?
CAMRYS
9
d-dimensional online algorithm

• Choose u 2 ltd, u1
• Choose xt1 xt ? u ft(xt?u)/?
• Repeat

x3
x4
x1
x2
S
10
Hidden complication

• Dealing with steps outside set is difficult!

S
11
Hidden complication

• Dealing with steps outside set is difficult!

S
reshape into Isotropic position
12
Related work conclusions

• Related work
• Online convex gradient descent Zinkevich03
• One point gradient estimates Spall97,Granichin89
  
• Same exact problem Kleinberg04 (different
  solution)
• Conclusions
• Can estimate gradient of a function from single
  evaluation (using randomness)
• Adaptive adversary ft(xt)ft(x1,x2,,xt)
• Useful for situations with a sequence of
  different functions, no gradient info, one
  evaluation per function, and others