CSCI6370: Topics in Computer Science Advanced Topics in Algorithms and Applications Fall Semester, 2

1
055133 Graph Algorithms and Combinatorial
Optimization Spring 2006Instructor Xiaodong
Wu
2
Textbooks

• Required
• T. Cormen, C. Leiserson, R. Rivest, and C. Stein,
  Introduction to Algorithms, 2nd edition, McGraw
  Hill, 2001.
• Recommended
• R. K. Ahuja, T.L. Magnanti, and J. B. Orlin,
  Network Flows, Theory, Algorithms, and
  Applications,  Prentice Hall, 1993
• Garey and Johnson, Computers and Intractability -
  A Guide to the Theory of NP-Completeness,
  Freeman, 1979.
• Alfred V. Aho John E. Hopcroft Jeffrey Ullman,
  Data Structures and Algorithms, Addison Wesley,
  1983.

3
Tentative Syllabus

• Introduces basic elements of the design and
  analysis of computer algorithms
• Quick review of basic data structures
• Analysis of algorithms, big-Oh notation,
  recurrences
• Classic algorithm design techniques
  divide-and-conquer, greedy methods, and dynamic
  programming
• Graph algorithms DFS, BFS, MST, shortest paths
• Network flows
• Computational complexity NP-completeness
• Approximation algorithms

4
Prerequisites

• 055033 - Introduction to Software Design
• Basic data structures
• lists, stacks, queues
• Trees
• heaps
• Basic algorithms for sorting and searching

5
Grading Policy

• Homework ( 7) 30
• Two Exams 25 each
• Term Project 20
• Each component will receive a numerical score.
  The course grade will be based on the weighted
  total of all components.

6
Some Regulations

• If you must miss an exam, make prior arrangements
  with the instructor.
• An F will be given to any student who fails to
  take exams.
• No late work will be accepted after the papers
  have been graded and returned.
• A one time late penalty of 50 will be assigned.
• No cheating.

7
Important Dates

• March 1 (Wednesday) Exam 1 (In Class)
• March 10 (Friday) Term Project Proposal
• April 5 (Wednesday) Exam 2 (In Class)
• May 10 (Wednesday) Term Project

8
Teaching Assistant

• Name Yuan Cai
• Office Room 1313 SC
• Email yucai_at_engineering.uiowa.edu
• Office Hours TR 1230pm - 200pm

9
What are algorithms studying about?
A problem from applications
Question 1 Is this problem solvable by
computers at all? (Complexity)
(yes) decidable (continue)
(no) undecidable (stop)
Question 2 Is this problem very hard to solve
on computers?
(no) find an efficient algorithm for it
(sequential or parallel )
(yes) show evidence for this conclusion
(NP-complete, NP- hard )
Develop algorithms for approximation solutions
10
Why bother to study algorithms?

• The problem Maximum contiguous subsequence sum
  problem
• Given a set of N real numbers A1, A2, , AN, find
  (and identify the sequence corresponding to) the
  maximum value of (Ai Ai1 Aj ).
• The maximum contiguous subsequence sum is zero if
  all the integers are negative. (Why?)
• Examples (maximum subsequences are underlined)
• -2, 11, -4, 13, -4, 2
• 1, -3, 4, -2, -1, 6

11
Algorithm 1 Brute Force Algorithm
i is the left index of a subsequence
j is the right index of a subsequence
Compute the total sum of the subsequence ai..j
If the total sum of the current subsequence is
large than the maximum sum so far obtained, then
set the maximum sum to be thisSum

• Consider all the subsequences and compute the
  total sum for each subsequence.
• How many?

12
Subsequence Generation in the Brute Force
Algorithm
a1
a2
a3
a4
a5
a0
j increases
i0
j increases
i1
j increases
i2
j increases
i3
i4
i5
The number of subsequences N (N-1) (N-2)
1 N(N1)/2
13
Running Time for Algorithm 1

• There are three nested loops. Each is of size at
  most N. So, the running time is O(N3).
• Slight over-estimate (a factor of 6) that results
  from some loops being of size less than N.
• More precise calculation of the size of the loops
  shows that the running time is ?(N3).

14
How to Improve

• If we can remove a loop, of course, we can reduce
  the running time. But it is not always possible.
• The innermost loop is unnecessary in the brute
  force algorithm.
• thisSum for next j is easily obtained from old
  value of thisSum
• In order to compute Ai A i1 A j-1 Aj
• Just computed Ai A i1 A j-1
• What we need is what we just computed Aj
• We only need one unit time instead of (j-i1)
  units time.

a1
a2
a3
a4
a5
a0
j increases
i0
15
Algorithm 2 Improved Algorithm
thisSum is obtained from the previously computed
thisSum.

• Running time Two nested loops each with size of
  at most N.
• T(N) O(N2)

16
Recursive Algorithm

• Use a divide-and-conquer strategy.
• The maximum subsequence either
• Case 1 lies entirely in the first half
• Case 2 lies entirely in the second half
• Case 3 starts somewhere in the first half, goes
  to the last element in the first half, continues
  at the first element in the second half, ends
  somewhere in the second half.
• Compute all three possibilities, and choose the
  maximum.
• First two possibilities easily computed
  recursively.

17
Computing Case 3

• Easily done with two loops.
• For maximum sum that starts in the first half and
  extends to the last element in the first half,
  use a right-to-left scan starting at the last
  element in the first half.
• For the other maximum sum, do a left-to-right
  scan, starting at the first element in the second
  half.

Second Half
First Half
4 -3 5 -2 -1 2 6 -2
4 0 3 -2 -1 1 7 5
18
Algorithm 3 Recursive Algorithm
Compute the maximum subsequence sum in the first
half.
Compute the maximum subsequence sum I n the
second half.
Use a right-to-left scan starting at the last
element in the first half to compute the
maximum sum that starts in the first half and
extends to the last element in the first half.
Use a left-to-right scan starting at the first
element in the second half to compute the
maximum sum in the second half that touches the
center divider.
Return the maximum among these three.
19
Running Time for Algorithm 3

• Let T(N) be the time for the recursive algorithm
  to solve the maximum subsequence sum problem of
  size N.
• T(N) 2 T(N / 2) N
• Two recursive calls, each of size N / 2. The time
  to solve each recursive call is T(N / 2) by the
  above definition
• Case 3 takes O(N) time we use N, because we will
  throw out the constants eventually.

• T(N) O(NlogN) (apply any one of the approaches
  we talked in the last lecture to solve this
  recurrence)

20
Algorithm 4 Linear-Time Algorithm

• Key observations
• If ai lt 0, then it cannot be the start of the
  optimal subsequence (since any subsequence that
  begins by including ai would be improved by
  beginning with ai1).
• Similarly, any negative subsequence cannot be a
  prefix of the optimal subsequence.
• Exmaple 1, -3, 4, -2, -1, 6, -3, 2

21
Algorithm 4 Linear-Time Algorithm
Start to generate a new subsequence.

• Running time One loop with size of at most N.
  T(N) O(N)

22
Extensions of Maximum subsequence sum problem

• Given an array A1..n of n real numbers and an
  integer k gt 0, find at most k contiguous
  subsequences that are pair-wise disjoint and
  whose total sum is maximized.
• Exmaple 1, -3, 4, -2, -1, 6, -1, 2
• k 4 1, -3, 4, -2, -1, 6, -1, 2
• k 2 1, -3, 4, -2, -1, 6, -1, 2
• k 1 1, -3, 4, -2, -1, 6, -1, 2
• Design an O(kn) algorithm.