ITCS6114: Algorithms and Data Structures - PowerPoint PPT Presentation

1 / 55
About This Presentation
Title:

ITCS6114: Algorithms and Data Structures

Description:

ITCS6114: Algorithms and Data Structures Introduction Proof by Induction Asymptotic Notation David Luebke * * David Luebke ... – PowerPoint PPT presentation

Number of Views:88
Avg rating:3.0/5.0
Slides: 56
Provided by: DrR61
Category:

less

Transcript and Presenter's Notes

Title: ITCS6114: Algorithms and Data Structures


1
ITCS6114 Algorithms and Data Structures
  • Introduction
  • Proof by Induction
  • Asymptotic Notation

2
The Course
  • Purpose a rigorous introduction to the design
    and analysis of algorithms
  • Not a lab or programming course
  • Not a math course, either
  • Suggested Textbooks
  • Introduction to The Design Analysis of
    Algorithms, Anany Levitin
  • Introduction to Algorithms, Cormen, Leiserson,
    Rivest, Stein

3
The Course
  • Instructor Zbigniew W. Ras
  • ras_at_uncc.edu
  • Office Woodward Hall 430C
  • Office hours Tuesday 1200-130pm
  • TA Ayman Hajja     
  • ahajja_at_uncc.edu Office Woodward Hall 404
    (Future Computing Lab)
  • Office hours Tuesday, Thursday 1130-130pm

4
The Course
  • Grading policy
  • Midterm 30 points
  • Project 30 points
  • Final 30 points
  • Participation 10 points
  • Grade A from 86 to 100 points, Grade B from 71 to
    85 points, Grade C from 56 to 70.

5
Review Induction
  • Suppose
  • S(k) is true for fixed constant k
  • Often k 0
  • S(n) ? S(n1) for all n gt k
  • Then S(n) is true for all n gt k

6
Proof By Induction
  • Claim S(n) is true for all n gt k
  • Basis
  • Show formula is true when n k
  • Inductive hypothesis
  • Assume formula is true for an arbitrary n
  • Step
  • Show that formula is then true for n1

7
Induction Example Gaussian Closed Form
  • Prove 1 2 3 n n(n1) / 2
  • Basis
  • If n 0, then 0 0(01) / 2
  • Inductive hypothesis
  • Assume 1 2 3 n n(n1) / 2
  • Step (show true for n1)
  • 1 2 n n1 (1 2 n) (n1)
  • n(n1)/2 n1 n(n1) 2(n1)/2
  • (n1)(n2)/2 (n1)(n1 1) / 2

8
Induction ExampleGeometric Closed Form
  • Prove a0 a1 an (an1 - 1)/(a - 1) for
    all a ? 1
  • Basis show that a0 (a01 - 1)/(a - 1)
  • a0 1 (a1 - 1)/(a - 1)
  • Inductive hypothesis
  • Assume a0 a1 an (an1 - 1)/(a - 1)
  • Step (show true for n1)
  • a0 a1 an1 a0 a1 an an1
  • (an1 - 1)/(a - 1) an1 (an11 - 1)/(a - 1)

9
Induction
  • Weve been using weak induction
  • Strong induction also holds
  • Basis show S(0)
  • Hypothesis assume S(k) holds for arbitrary k lt
    n
  • Step Show S(n1) follows
  • Another variation
  • Basis show S(0), S(1)
  • Hypothesis assume S(n) and S(n1) are true
  • Step show S(n2) follows

10
Asymptotic Performance
  • In this course, we care most about asymptotic
    performance
  • How does the algorithm behave as the problem size
    gets very large?
  • Running time
  • Memory/storage requirements

11
Asymptotic Notation
  • By now you should have an intuitive feel for
    asymptotic (big-O) notation
  • What does O(n) running time mean? O(n2)?O(n lg
    n)?
  • How does asymptotic running time relate to
    asymptotic memory usage?
  • Our first task is to define this notation more
    formally and completely

12
Input Size
  • Time and space complexity
  • This is generally a function of the input size
  • E.g., sorting, multiplication
  • How we characterize input size depends
  • Sorting number of input items
  • Multiplication total number of bits
  • Graph algorithms number of nodes edges
  • Etc

13
Analysis
  • Worst case
  • Provides an upper bound on running time
  • An absolute guarantee
  • Average case
  • Provides the expected running time
  • Very useful, but treat with care what is
    average?
  • Random (equally likely) inputs
  • Real-life inputs

14
An Example Insertion Sort
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 14
8/3/2015
15
An Example Insertion Sort
i ? j ? key ?Aj ? Aj1 ?
30
10
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 15
8/3/2015
16
An Example Insertion Sort
i 2 j 1 key 10Aj 30 Aj1 10
30
10
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 16
8/3/2015
17
An Example Insertion Sort
i 2 j 1 key 10Aj 30 Aj1 30
30
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 17
8/3/2015
18
An Example Insertion Sort
i 2 j 1 key 10Aj 30 Aj1 30
30
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 18
8/3/2015
19
An Example Insertion Sort
i 2 j 0 key 10Aj ? Aj1 30
30
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 19
8/3/2015
20
An Example Insertion Sort
i 2 j 0 key 10Aj ? Aj1 30
30
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 20
8/3/2015
21
An Example Insertion Sort
i 2 j 0 key 10Aj ? Aj1 10
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 21
8/3/2015
22
An Example Insertion Sort
i 3 j 0 key 10Aj ? Aj1 10
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 22
8/3/2015
23
An Example Insertion Sort
i 3 j 0 key 40Aj ? Aj1 10
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 23
8/3/2015
24
An Example Insertion Sort
i 3 j 0 key 40Aj ? Aj1 10
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 24
8/3/2015
25
An Example Insertion Sort
i 3 j 2 key 40Aj 30 Aj1 40
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 25
8/3/2015
26
An Example Insertion Sort
i 3 j 2 key 40Aj 30 Aj1 40
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 26
8/3/2015
27
An Example Insertion Sort
i 3 j 2 key 40Aj 30 Aj1 40
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 27
8/3/2015
28
An Example Insertion Sort
i 4 j 2 key 40Aj 30 Aj1 40
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 28
8/3/2015
29
An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 40
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 29
8/3/2015
30
An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 40
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 30
8/3/2015
31
An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 20
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 31
8/3/2015
32
An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 20
10
30
40
20
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 32
8/3/2015
33
An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 40
10
30
40
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 33
8/3/2015
34
An Example Insertion Sort
i 4 j 3 key 20Aj 40 Aj1 40
10
30
40
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 34
8/3/2015
35
An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 40
10
30
40
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 35
8/3/2015
36
An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 40
10
30
40
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 36
8/3/2015
37
An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 30
10
30
30
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 37
8/3/2015
38
An Example Insertion Sort
i 4 j 2 key 20Aj 30 Aj1 30
10
30
30
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 38
8/3/2015
39
An Example Insertion Sort
i 4 j 1 key 20Aj 10 Aj1 30
10
30
30
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 39
8/3/2015
40
An Example Insertion Sort
i 4 j 1 key 20Aj 10 Aj1 30
10
30
30
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 40
8/3/2015
41
An Example Insertion Sort
i 4 j 1 key 20Aj 10 Aj1 20
10
20
30
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
David Luebke 41
8/3/2015
42
An Example Insertion Sort
i 4 j 1 key 20Aj 10 Aj1 20
10
20
30
40
1
2
3
4
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
Done!
David Luebke 42
8/3/2015
43
Animating Insertion Sort
  • Check out the Animator, a java applet at
    http//www.sorting-algorithms.com/insertion-sort
  • Try it out with random, ascending, and descending
    inputs

David Luebke 43
8/3/2015
44
Insertion Sort
InsertionSort(A, n) for i 2 to n key
Ai j i - 1 while (j gt 0) and (Aj gt key)
Aj1 Aj j j - 1 Aj1
key
How many times will this loop execute?
David Luebke 44
8/3/2015
45
Analysis
  • Simplifications
  • Ignore actual and abstract statement costs
  • Order of growth is the interesting measure
  • Highest-order term is what counts
  • Remember, we are doing asymptotic analysis
  • As the input size grows larger it is the high
    order term that dominates

David Luebke 45
8/3/2015
46
Upper Bound Notation
  • We say InsertionSorts run time is O(n2)
  • Properly we should say run time is in O(n2)
  • Read O as Big-O (youll also hear it as
    order)
  • In general a function
  • f(n) is O(g(n)) if there exist positive constants
    c and n0 such that f(n) ? c ? g(n) for all n ? n0
  • Formally
  • O(g(n)) f(n) ? positive constants c and n0
    such that f(n) ? c ? g(n) ? n ? n0

David Luebke 46
8/3/2015
47
Insertion Sort Is O(n2)
  • Proof
  • Suppose runtime is an2 bn c
  • If any of a, b, and c are less than 0 replace
    the constant with its absolute value
  • an2 bn c ? (a b c)n2 (a b c)n (a
    b c)
  • ? 3(a b c)n2 for n ? 1
  • Let c 3(a b c) and let n0 1
  • Question
  • Is InsertionSort O(n3)?
  • Is InsertionSort O(n)?

David Luebke 47
8/3/2015
48
Lower Bound Notation
  • We say InsertionSorts run time is ?(n)
  • In general a function
  • f(n) is ?(g(n)) if ? positive constants c and n0
    such that 0 ? c?g(n) ? f(n) ? n ? n0
  • Proof
  • Suppose run time is an b
  • Assume a and b are positive (what if b is
    negative?)
  • an ? an b

David Luebke 48
8/3/2015
49
Asymptotic Tight Bound
  • A function f(n) is ?(g(n)) if ? positive
    constants c1, c2, and n0 such that c1 g(n) ?
    f(n) ? c2 g(n) ? n ? n0
  • Theorem
  • f(n) is ?(g(n)) iff f(n) is both O(g(n)) and
    ?(g(n))

David Luebke 49
8/3/2015
50
Practical Complexity
David Luebke 50
8/3/2015
51
Practical Complexity
David Luebke 51
8/3/2015
52
Practical Complexity
David Luebke 52
8/3/2015
53
Practical Complexity
David Luebke 53
8/3/2015
54
Practical Complexity
David Luebke 54
8/3/2015
55
Other Asymptotic Notations
  • A function f(n) is o(g(n)) if ? positive
    constants c and n0 such that f(n) lt c g(n) ? n
    ? n0
  • A function f(n) is ?(g(n)) if ? positive
    constants c and n0 such that c g(n) lt f(n) ? n
    ? n0
  • Intuitively,
  • o() is like lt
  • O() is like ?
  • ?() is like gt
  • ?() is like ?
  • ?() is like

David Luebke 55
8/3/2015
Write a Comment
User Comments (0)
About PowerShow.com