CSE 960

1
CSE 960

• Spring 2005

2
Outline

• Logistical details
• Presentation evaluations
• Scheduling
• Approximation Algorithms
• Search space view of mathematical concepts
• Dynamic programming

3
Course Overview

• We will be looking at a collection of
  mathematical ideas and how they influence current
  research in algorithms
• Mathematical programming
• Game theory
• Applications to scheduling

4
Other Points of Emphasis

• Presentation skills
• We will develop metrics for evaluating
  presentations to be given later
• Group work
• I hope to make class periods interactive with you
  working in informal groups to make sure all
  understand the material being presented

5
Caveats

• We will not be doing an in-depth examination of
  any idea instead, we will get exposure to the
  concepts so that you are prepared to learn more
  on your own
• I am NOT an expert in these areas and am teaching
  them because I need to learn them better
• Emphasis on scheduling applications because that
  is where my research interest lie

6
Grading

• Presentations 50
• 1 group presentation of a textbook chapter
• 1 individual presentation of a scientific paper
• Homework 25
• Short assignments, each question 1 point
• Class Participation/group work 25

7
Outline

• Logistical details
• Presentation evaluations
• Scheduling
• Approximation Algorithms
• Search space view of mathematical concepts
• Dynamic programming

8
How should we evaluate a presentation?

• Group discussion of presentation evaluations
• Create a safe environment for all to
  participate
• Stay on task
• Recorder role
• Present to class

9
Outline

• Logistical details
• Presentation evaluations
• Scheduling
• Approximation Algorithms
• Search space view of mathematical concepts
• Dynamic programming

10
Scheduling

• The problem of assigning jobs to machines to
  minimize some objective function
• 3 parameter notation
• machine environment job characteristic obj
• Machine environments 1, P, Q, R
• Job characteristics preemption, release dates,
  weights, values, deadlines
• Objectives makespan, average completion time,
  average flow time, etc.

11
Example Problem

• 2 max Cj
• 2 machines, no preemptions, no release dates,
  goal is to minimize the maximum completion time
  of any job
• Example input jobs with length 1, 1, 2
• What might be an obvious greedy algorithm for
  this problem?
• Argue why this problem is NP-hard

12
Outline

• Logistical details
• Presentation evaluations
• Scheduling
• Approximation Algorithms
• Search space view of mathematical concepts
• Dynamic programming

13
Approximation Algorithms

• Many problems we study will be NP-hard
• In such cases, we desire to have polynomial-time
  approximation algorithms
• A(I)/OPT(I) c for some constant c (min
  objective)
• Algorithm runs in polynomial time in n, the
  problem size
• Approximation algorithm for makespan scheduling?

14
PTAS

• Even better, we like to have polynomial-time
  approximation schemes (PTAS)
• A(I, e)/OPT(I) (1 e) for e 0 (min objective)
• Running time is polynomial in n, the problem
  size, but may be exponential in 1/ e
• Even better is if we can be polynomial in 1/ e
  too
• Often such schemes are not very practical, but
  are theoretically desirable
• PTAS for makespan scheduling?

15
Outline

• Logistical details
• Presentation evaluations
• Scheduling
• Approximation Algorithms
• Search space view of mathematical concepts
• Dynamic programming

16
Searching for Optimal Solution

• All topics may be viewed as searching a space of
  solutions for an optimal solution
• Dynamic programming
• discrete search space
• Recursive solution structure
• Mathematical programming
• Discrete/Continuous search space
• Constraint-based solution structure
• Game Theory
• Discrete/mixed search space
• Multiple selfish players

17
Dynamic Programming
18
Mathematical Programming
Y
X
19
Game Theory
Player A Move
Player B Move
20
Outline

• Logistical details
• Presentation evaluations
• Scheduling
• Approximation Algorithms
• Search space view of mathematical concepts
• Dynamic programming

21
Overview

• The key idea behind dynamic program is that it is
  a divide-and-conquer technique at heart
• That is, we solve larger problems by patching
  together solutions to smaller problems
• However, dynamic programming is typically faster
  because we compute these solutions in a bottom-up
  fashion

22
Fibonacci numbers

• F(n) F(n-1) F(n-2)
• F(0) 0
• F(1) 1
• Top-down recursive computation is very
  inefficient
• Many F(i) values are computed multiple times
• Bottom-up computation is much more efficient
• Compute F(2), then F(3), then F(4), etc. using
  stored values for smaller F(i) values to compute
  next value
• Each F(i) value is computed just once

23
Recursive Computation
F(n) F(n-1) F(n-2) F(0) 0, F(1)
1 Recursive Solution
24
Bottom-up computation
We can calculate F(n) in linear time by storing
small values. F0 0 F1 1 for i 2 to
n Fi Fi-1 Fi-2 return Fn Moral We
can sometimes trade space for time.
25
Key implementation steps

• Identify subsolutions that may be useful in
  computing whole solution
• Often need to introduce parameters
• Develop a recurrence relation (recursive
  solution)
• Set up the table of values/costs to be computed
• The dimensionality is typically determined by the
  number of parameters
• The number of values should be polynomial
• Determine the order of computation of values
• Backtrack through the table to obtain complete
  solution (not just solution value)

26
Example Matrix Multiplication

• Input
• List of n matrices to be multiplied together
  using traditional matrix multiplication
• The dimensions of the matrices are sufficient
• Task
• Compute the optimal ordering of multiplications
  to minimize total number of scalar
  multiplications performed
• Observations
• Multiplying an X ? Y matrix by a Y ? Z matrix
  takes X ? Y ? Z multiplications
• Matrix multiplication is associative but not
  commutative

27
Example Input

• Input
• M1, M2, M3, M4
• M1 13 x 5
• M2 5 x 89
• M3 89 x 3
• M4 3 x 34
• Feasible solutions and their values
• ((M1 M2) M3) M410,582 scalar multiplications
• (M1 M2) (M3 M4) 54,201 scalar multiplications
• (M1 (M2 M3)) M4 2856 scalar multiplications
• M1 ((M2 M3) M4) 4055 scalar multiplications
• M1 (M2 (M3 M4)) 26,418 scalar multiplications

28
Identify subsolutions

• Often need to introduce parameters
• Define dimensions to be (d0, d1, , dn) where
  matrix Mi has dimensions di-1 x di
• Let M(i,j) be the matrix formed by multiplying
  matrices Mi through Mj
• Define C(i,j) to be the minimum cost for
  computing M(i,j)

29
Develop a recurrence relation

• Definitions
• M(i,j) matrices Mi through Mj
• C(i,j) the minimum cost for computing M(i,j)
• Recurrence relation for C(i,j)
• C(i,i) ???
• C(i,j) ???
• Want to express C(i,j) in terms of smaller C
  terms

30
Set up table of values

• Table
• The dimensionality is typically determined by the
  number of parameters
• The number of values should be polynomial

31
Order of Computation of Values

• Many orders are typically ok.
• Just need to obey some constraints
• What are valid orders for this table?

32
Representing optimal solution
P(i,j) records the intermediate multiplication k
used to compute M(i,j). That is, P(i,j) k if
last multiplication was M(i,k) M(k1,j)
33
Pseudocode
int MatrixOrder() forall i, j Ci, j 0 for j
2 to n for i j-1 to 1 C(i,j) mini(C(i,k) C(k1,j) di-1dkdj) Pi, jk return
C1, n
34
Backtracking
Procedure ShowOrder(i, j) if (ij) write ( Ai)
else k P i, j write (
ShowOrder(i, k) write ? ShowOrder (k1,
j) write )
35
Principle of Optimality

• In book, this is termed Optimal substructure
• An optimal solution contains within it optimal
  solutions to subproblems.
• More detailed explanation
• Suppose solution S is optimal for problem P.
• Suppose we decompose P into P1 through Pk and
  that S can be decomposed into pieces S1 through
  Sk corresponding to the subproblems.
• Then solution Si is an optimal solution for
  subproblem Pi

36
Outline

• Logistical details
• Presentation evaluations
• Scheduling
• Approximation Algorithms
• Search space view of mathematical concepts
• Dynamic programming
• Extra notes on dynamic programming

37
Example 1

• Matrix Multiplication
• In our solution for computing matrix M(1,n), we
  have a final step of multiplying matrices M(1,k)
  and M(k1,n).
• Our subproblems then would be to compute M(1,k)
  and M(k1,n)
• Our solution uses optimal solutions for computing
  M(1,k) and M(k1,n) as part of the overall
  solution.

38
Example 2

• Shortest Path Problem
• Suppose a shortest path from s to t visits u
• We can decompose the path into s-u and u-t.
• The s-u path must be a shortest path from s to u,
  and the u-t path must be a shortest path from u
  to t
• Conclusion dynamic programming can be used for
  computing shortest paths

39
Example 3

• Longest Path Problem
• Suppose a longest path from s to t visits u
• We can decompose the path into s-u and u-t.
• Is it true that the s-u path must be a longest
  path from s to u?
• Conclusion?

40
Example 4 The Traveling Salesman Problem
What recurrence relation will return the optimal
solution to the Traveling Salesman Problem? If
T(i) is the optimal tour on the first i points,
will this help us in solving larger instances of
the problem? Can we set T(i1) to be T(i) with
the additional point inserted in the position
that will result in the shortest path?
41
No!
42
Summary of bad examples

• There almost always is a way to have the optimal
  substructure if you expand your subproblems
  enough
• For longest path and TSP, the number of
  subproblems grows to exponential size
• This is not useful as we do not want to compute
  an exponential number of solutions

43
When is dynamic programming effective?

• Dynamic programming works best on objects that
  are linearly ordered and cannot be rearranged
• characters in a string
• files in a filing cabinet
• points around the boundary of a polygon
• the left-to-right order of leaves in a search
  tree.
• Whenever your objects are ordered in a
  left-to-right way, dynamic programming must be
  considered.

44
Efficient Top-Down Implementation

• We can implement any dynamic programming solution
  top-down by storing computed values in the table
• If all values need to be computed anyway, bottom
  up is more efficient
• If some do not need to be computed, top-down may
  be faster

45
Trading Post Problem

• Input
• n trading posts on a river
• R(i,j) is the cost for renting at post i and
  returning at post j for i
• Note, cannot paddle upstream so i
• Task
• Output minimum cost route to get from trading
  post 1 to trading post n

46
Longest Common Subsequence Problem

• Given 2 strings S and T, a common subsequence is
  a subsequence that appears in both S and T.
• The longest common subsequence problem is to find
  a longest common subsequence (lcs) of S and T
• subsequence characters need not be contiguous
• different than substring
• Can you use dynamic programming to solve the
  longest common subsequence problem?

47
Longest Increasing Subsequence Problem

• Input a sequence of n numbers x1, x2, , xn.
• Task Find the longest increasing subsequence of
  numbers
• subsequence numbers need not be contiguous
• Can you use dynamic programming to solve the
  longest common subsequence problem?

48
Book Stacking Problem

• Input
• n books with heights hi and thicknesses ti
• length of shelf L
• Task
• Assignment of books to shelves minimizing sum of
  heights of tallest book on each shelf
• books must be stored in order to conform to
  catalog system (i.e. books on first shelf must be
  1 through i, books on second shelf i1 through k,
  etc.)