Complexity

1
Complexity

• COMP 114
• Thursday February 28

2
Announcements

• Reading assignment to follow

3
Topics

• Last Time
• Recursion

• This Class
• Caching of results
• Complexity

4
Fibonacci Numbers

• Simple series of numbers
• 1 1 2 3 5 8 13 21 34 55 89
• Just sum of previous two numbers.
• Computed recursively for n gt 0 as

5
Simple Program

• Lets look at example Fib1

6
What Can We Do?

• Any ideas how to speed it up?
• Hint Look for redundancy
• Look at Fib2

7
Caching of Results

• General Technique
• Useful not only for recursion.
• Put it in your toolkit.

8
Todays Goals

• Informal study of program performance
• like Towers of Hanoi
• Soon well be studying / comparing algorithms for
• searching
• sorting
• More formal treatment in COMP121 and 122

9
Measures of Performance

• First thing that comes to mind is time.
• From almost no time
• to 500 billion years

• However, space is often as important
• We have a machine with 16 GB memory
• Still run out of space!

10
Measuring Running Time

• Easy to do
• One thing we can do is measure clock time.
• Look at FibTimed

11
Clock Time

• Not a very good measure
• Affected by other programs
• Affected by I/O
• You can get CPU time instead.
• I havent tried in Java

12
Is Running Time What We Want?

• It tells us something about a particular program
• We can tune for performance
• However, we saw that what really helped was not
  tuning
• It was changing algorithm to use cache

13
Asymptotic Behavior

• We will use an approximate measure of the amount
  of work as the problem scales up.
• A function of problem size
• Like how many rings on Towers
• Measures algorithms
• not implementations
• This is often called complexity

14
Linear Complexity

• Consider a problem like this
• for(i 0 i lt n i)
• result i
• Inner part of loop can be
• any constant work.
• Would be called O(n)
• Big-O notation

15
O(n2)

• How about this?
• for(i 0 i lt n i)
• for(j 0 j lt n j)
• result i j
• Now theres more work as n increases.
• For every iteration of outer loop were doing n
  pieces of work

16
How Much Work?
17
Fibonacci
18
Big-O

• Notation we use is O(n)
• or O(n2), O(2n)
• Called Big-O notation
• Stands for on the order of
• Read as Big-O n or Order n
• Formal definition in a moment

19
Problem Size

• First thing is to decide what to use to measure
  problem size
• Sometimes easy
• Items to be sorted
• Rings in Towers of Hanoi
• Matrix usually n, one side, instead of n2
• Sometimes two sizes m and n

20
Operations

• What basic operations must be performed?
• Comparison for the sort
• Moving a ring
• Adding elements
• Whats the complexity of the basic operation?
• Is it constant?
• If not, increases complexity.

21
Formal Definition

• Let f and g be non-negative, non-decreasing
  functions of n.
• Then f is O(g) if
• there exist positive constants c and n0
• Such that f(n) ? c g(n) for all n gt n0.

22
Ignore Constant Factor

• Why?
• Because as the program gets bigger, it will only
  change the running time by a constant factor.
• We can always find a bigger problem size that
  makes constant factor meaningless.
• In reality, of course, it does matter (small n).

23
Common Complexities

• O(n)
• O(n2)
• O(np)
• O(2n)
• O(1)
• Well see others in a moment

24
Possible Measures

• Best Case
• Average Case
• Worst-Case

25
Search

• Lets say we want to find an item in an array.
• search(int target, int array)
• Lets try simple way first.

26
Linear Search

• public static int search(int t, int a)
• for(int i 0 i lt a.length i)
• if(ai t)
• return i
• return 1
• How should we decide complexity?

27
Worst Case

• public static int search(int t, int a)
• for(int i 0 i lt a.length i)
• if(ai t)
• return i
• return 1
• Whats worst case run time?
• When does it occur?

28
Best Case

• public static int search(int t, int a)
• for(int i 0 i lt a.length i)
• if(ai t)
• return i
• return 1
• Whats best case run time?
• When does it occur?

29
Average Case

• public static int search(int t, int a)
• for(int i 0 i lt a.length i)
• if(ai t)
• return i
• return 1
• Whats average?

30
Linear Search

• Its O(n)
• For large cases, the fact that average is n/2
  doesnt matter much
• Well see in next example
• But first, review

31
Review Search for Objects

• public static int search(Object target,
• Object array)
• for(int i 0 i lt array.length i)
• if(arrayi.equals(target))
• return i
• return 1

32
Search in Sorted List

• How do you search in phone book or dictionary?
• Computing equivalent is Binary Search

33
Binary Search
1
2
3
4
5
6
7
8
9
1
2
3
4
6
7
8
9
6
8
9
1
3
4
9
4
34
Example Search for 3
1
2
3
4
5
6
7
8
9
35
Code

• public static int binarySearch(int target,
  int array, int size)
• int low 0, middle 0, high size - 1
• while (low lt high)
• middle (low high) / 2
• if (arraymiddle target)
• return middle
• else if (arraymiddle lt target)
• low middle 1
• else
• high middle - 1
• return - 1

36
What is Complexity?

• Worst Case
• Lets look at search for 10 again.

37
Example Search for 10
9
38
How much work?

• Divide the list
• n / 2 / 2 / 2 until size is 1
• or
• n / 2 / 2 / 2 1
• Say the division happens k times
• So n / 2k 1
• n 2k
• k log2n

39
Binary Search

• O(log n)

Why binary?
k is number of bits to represent n
40
How about for Objects?

• Implement a compareTo( ) method
• You have to be able to determine
• less than, equal, or greater than
• You have to do that for sort also
• Usually its alphabetical order

41
Summary

• Characterize algorithms by order of complexity
• Measure work as size of problem increases
• Logarithmic, constant are good
• Polynomial starts to get slow
• Exponential is intractable

42
Next Time

• Mathematical properties of programs
• After Spring Break
• Sorting