Title: ITCS6114: Algorithms and Data Structures
1ITCS6114 Algorithms and Data Structures
- Introduction
- Proof by Induction
- Asymptotic Notation
2The Course
- Purpose a rigorous introduction to the design
and analysis of algorithms - Not a lab or programming course
- Not a math course, either
- Suggested Textbooks
- Introduction to The Design Analysis of
Algorithms, Anany Levitin - Introduction to Algorithms, Cormen, Leiserson,
Rivest, Stein
3The Course
- Instructor Zbigniew W. Ras
- ras_at_uncc.edu
- Office Woodward Hall 430C
- Office hours Tuesday 1200-130pm
- TA Ayman Hajja
- ahajja_at_uncc.edu Office Woodward Hall 404
(Future Computing Lab) - Office hours Tuesday, Thursday 1130-130pm
-
4The Course
- Grading policy
- Midterm 30 points
- Project 30 points
- Final 30 points
- Participation 10 points
- Grade A from 86 to 100 points, Grade B from 71 to
85 points, Grade C from 56 to 70.
5Review Induction
- Suppose
- S(k) is true for fixed constant k
- Often k 0
- S(n) ? S(n1) for all n gt k
- Then S(n) is true for all n gt k
6Proof By Induction
- Claim S(n) is true for all n gt k
- Basis
- Show formula is true when n k
- Inductive hypothesis
- Assume formula is true for an arbitrary n
- Step
- Show that formula is then true for n1
7Induction Example Gaussian Closed Form
- Prove 1 2 3 n n(n1) / 2
- Basis
- If n 0, then 0 0(01) / 2
- Inductive hypothesis
- Assume 1 2 3 n n(n1) / 2
- Step (show true for n1)
- 1 2 n n1 (1 2 n) (n1)
- n(n1)/2 n1 n(n1) 2(n1)/2
- (n1)(n2)/2 (n1)(n1 1) / 2
8Induction ExampleGeometric Closed Form
- Prove a0 a1 an (an1 - 1)/(a - 1) for
all a ? 1 - Basis show that a0 (a01 - 1)/(a - 1)
- a0 1 (a1 - 1)/(a - 1)
- Inductive hypothesis
- Assume a0 a1 an (an1 - 1)/(a - 1)
- Step (show true for n1)
- a0 a1 an1 a0 a1 an an1
- (an1 - 1)/(a - 1) an1 (an11 - 1)/(a - 1)
9Induction
- Weve been using weak induction
- Strong induction also holds
- Basis show S(0)
- Hypothesis assume S(k) holds for arbitrary k lt
n - Step Show S(n1) follows
- Another variation
- Basis show S(0), S(1)
- Hypothesis assume S(n) and S(n1) are true
- Step show S(n2) follows
10Asymptotic Performance
- In this course, we care most about asymptotic
performance - How does the algorithm behave as the problem size
gets very large? - Running time
- Memory/storage requirements
11Asymptotic Notation
- By now you should have an intuitive feel for
asymptotic (big-O) notation - What does O(n) running time mean? O(n2)?O(n lg
n)? - How does asymptotic running time relate to
asymptotic memory usage? - Our first task is to define this notation more
formally and completely
12Input Size
- Time and space complexity
- This is generally a function of the input size
- E.g., sorting, multiplication
- How we characterize input size depends
- Sorting number of input items
- Multiplication total number of bits
- Graph algorithms number of nodes edges
- Etc
13Analysis
- Worst case
- Provides an upper bound on running time
- An absolute guarantee
- Average case
- Provides the expected running time
- Very useful, but treat with care what is
average? - Random (equally likely) inputs
- Real-life inputs
14An Example Insertion Sort
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 14
8/3/2015
15An Example Insertion Sort
i ? j ? key ?Aj ? Aj1 ?
30
10
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 15
8/3/2015
16An Example Insertion Sort
i 2 j 1 key 10Aj 30 Aj1 10
30
10
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 16
8/3/2015
17An Example Insertion Sort
i 2 j 1 key 10Aj 30 Aj1 30
30
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 17
8/3/2015
18An Example Insertion Sort
i 2 j 1 key 10Aj 30 Aj1 30
30
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 18
8/3/2015
19An Example Insertion Sort
i 2 j 0 key 10Aj ? Aj1 30
30
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 19
8/3/2015
20An Example Insertion Sort
i 2 j 0 key 10Aj ? Aj1 30
30
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 20
8/3/2015
21An Example Insertion Sort
i 2 j 0 key 10Aj ? Aj1 10
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 21
8/3/2015
22An Example Insertion Sort
i 3 j 0 key 10Aj ? Aj1 10
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 22
8/3/2015
23An Example Insertion Sort
i 3 j 0 key 40Aj ? Aj1 10
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 23
8/3/2015
24An Example Insertion Sort
i 3 j 0 key 40Aj ? Aj1 10
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 24
8/3/2015
25An Example Insertion Sort
i 3 j 2 key 40Aj 30 Aj1 40
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 25
8/3/2015
26An Example Insertion Sort
i 3 j 2 key 40Aj 30 Aj1 40
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 26
8/3/2015
27An Example Insertion Sort
i 3 j 2 key 40Aj 30 Aj1 40
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 27
8/3/2015
28An Example Insertion Sort
i 4 j 2 key 40Aj 30 Aj1 40
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 28
8/3/2015
29An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 40
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 29
8/3/2015
30An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 40
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 30
8/3/2015
31An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 20
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 31
8/3/2015
32An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 20
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 32
8/3/2015
33An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 40
10
30
40
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 33
8/3/2015
34An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 40
10
30
40
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 34
8/3/2015
35An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 40
10
30
40
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 35
8/3/2015
36An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 40
10
30
40
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 36
8/3/2015
37An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 30
10
30
30
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 37
8/3/2015
38An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 30
10
30
30
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 38
8/3/2015
39An Example Insertion Sort
i 4 j 1 key 20Aj 10 Aj1 30
10
30
30
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 39
8/3/2015
40An Example Insertion Sort
i 4 j 1 key 20Aj 10 Aj1 30
10
30
30
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 40
8/3/2015
41An Example Insertion Sort
i 4 j 1 key 20Aj 10 Aj1 20
10
20
30
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 41
8/3/2015
42An Example Insertion Sort
i 4 j 1 key 20Aj 10 Aj1 20
10
20
30
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
Done!
David Luebke 42
8/3/2015
43Animating Insertion Sort
- Check out the Animator, a java applet at
http//www.sorting-algorithms.com/insertion-sort - Try it out with random, ascending, and descending
inputs
David Luebke 43
8/3/2015
44Insertion Sort
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
How many times will this loop execute?
David Luebke 44
8/3/2015
45Analysis
- Simplifications
- Ignore actual and abstract statement costs
- Order of growth is the interesting measure
- Highest-order term is what counts
- Remember, we are doing asymptotic analysis
- As the input size grows larger it is the high
order term that dominates
David Luebke 45
8/3/2015
46Upper Bound Notation
- We say InsertionSorts run time is O(n2)
- Properly we should say run time is in O(n2)
- Read O as Big-O (youll also hear it as
order) - In general a function
- f(n) is O(g(n)) if there exist positive constants
c and n0 such that f(n) ? c ? g(n) for all n ? n0 - Formally
- O(g(n)) f(n) ? positive constants c and n0
such that f(n) ? c ? g(n) ? n ? n0
David Luebke 46
8/3/2015
47Insertion Sort Is O(n2)
- Proof
- Suppose runtime is an2 bn c
- If any of a, b, and c are less than 0 replace
the constant with its absolute value - an2 bn c ? (a b c)n2 (a b c)n (a
b c) - ? 3(a b c)n2 for n ? 1
- Let c 3(a b c) and let n0 1
- Question
- Is InsertionSort O(n3)?
- Is InsertionSort O(n)?
David Luebke 47
8/3/2015
48Lower Bound Notation
- We say InsertionSorts run time is ?(n)
- In general a function
- f(n) is ?(g(n)) if ? positive constants c and n0
such that 0 ? c?g(n) ? f(n) ? n ? n0 - Proof
- Suppose run time is an b
- Assume a and b are positive (what if b is
negative?) - an ? an b
David Luebke 48
8/3/2015
49Asymptotic Tight Bound
- A function f(n) is ?(g(n)) if ? positive
constants c1, c2, and n0 such that c1 g(n) ?
f(n) ? c2 g(n) ? n ? n0 - Theorem
- f(n) is ?(g(n)) iff f(n) is both O(g(n)) and
?(g(n))
David Luebke 49
8/3/2015
50Practical Complexity
David Luebke 50
8/3/2015
51Practical Complexity
David Luebke 51
8/3/2015
52Practical Complexity
David Luebke 52
8/3/2015
53Practical Complexity
David Luebke 53
8/3/2015
54Practical Complexity
David Luebke 54
8/3/2015
55Other Asymptotic Notations
- A function f(n) is o(g(n)) if ? positive
constants c and n0 such that f(n) lt c g(n) ? n
? n0 - A function f(n) is ?(g(n)) if ? positive
constants c and n0 such that c g(n) lt f(n) ? n
? n0 - Intuitively,
- o() is like lt
- O() is like ?
- ?() is like gt
- ?() is like ?
David Luebke 55
8/3/2015