Algorithms and Analysis

1
Algorithms and Analysis

• CS 1428 Foundations of CS I

2
Algorithms

• Step-by-step instructions that tell a computing
  agent how to solve some problem using only finite
  resources
• Resources
• Memory
• CPU cycles
• Time/Space
• Types of instructions
• Sequential
• Conditional
• If statements
• Iterative
• Loops

3
Pseudocode The Interlingua for Algorithms

• an English-like description of the sequential,
  conditional, and iterative operations of an
  algorithm
• no rigid syntax. As with an essay, clarity and
  organization are key. So is completeness.

4
Pseudocode Example
Find Largest Number Input A list of positive
numbers Output The largest number in the list
Procedure 1. Set Largest to zero 2.
Set Current-Number to the first in the list
3. While there are more numbers in the list
3.1 if (the Current-Number gt Largest)
then 3.1.1 Set Largest
to the Current-Number 3.2 Set
Current-Number to the next one in the list
4. Output Largest
5
Pseudocode Example
Find Largest Number Input A list of positive
numbers Output The largest number in the list
Procedure 1. Set Largest to zero 2.
Set Current-Number to the first in the list
3. While there are more numbers in the list
3.1 if (the Current-Number gt Largest)
then 3.1.1 Set Largest
to the Current-Number 3.2 Set
Current-Number to the next one in the list
4. Output Largest
conditional operation
6
Pseudocode Example
Find Largest Number Input A list of positive
numbers Output The largest number in the list
Procedure 1. Set Largest to zero 2.
Set Current-Number to the first in the list
3. While there are more numbers in the list
3.1 if (the Current-Number gt Largest)
then 3.1.1 Set Largest
to the Current-Number 3.2 Set
Current-Number to the next one in the list
4. Output Largest
iterative operation
7
Pseudocode Example
Find Largest Number Input A list of positive
numbers Output The largest number in the list
Procedure 1. Set Largest to zero 2.
Set Current-Number to the first in the list
3. While there are more numbers in the list
3.1 if (the Current-Number gt Largest)
then 3.1.1 Set Largest
to the Current-Number 3.2 Set
Current-Number to the next one in the list
4. Output Largest
Lets play computer to review this algorithm
8
Algorithms vary in efficiency
example sum the numbers from 1 to n
efficiency space 3 memory cells time
t(step1) t(step 2)
n t(step 4) n t(step 5)

• space requirement is constant (i.e. independent
  of n)
• time requirement is linear (i.e. grows linearly
  with n). This is written O(n)?

to see this graphically...
9
Algorithm Is time requirements
The exact equation for the line is unknown
because we lack precise values for the constants
m and b. But, we can say time is a linear
function of the size of the problem time O(n)
10
Algorithm II for summation
First, consider a specific case n 100.
The key insight, due to Gauss the numbers can
be grouped into 50 pairs of the form 1 100
101 2 99 101 . . . 50
51 101
Second, generalize the formula for any (even) n
sum (n / 2) (n 1)
Time requirement is constant. time O(1)
11
Sequential Search A Commonly used Algorithm

• Suppose you want to a find a student in the Texas
  State directory.
• It contains EIDs, names, phone numbers, lots of
  other information.
• You want a computer application for searching the
  directory given an EID, return the students
  phone number.
• You want more, too, but this is a good start

12
Sequential Search of a student database
13
Time requirements for sequential search
because the amount of work is a constant multiple
of n, the time requirement is O(n) in the worst
case and the average case.
14
O(n) searching is too slow
Bad news for searching NYC phone book, IRS
database, ...
15
Searching an ordered list is faster an example
of binary search
16
The binary search algorithm
assuming that the entries in student are sorted
in increasing order,
1. ask user to input studentNum to search for 2.
set found to no 3. while not done searching
and found no 4. set middle to the
index counter at the middle of the student
list 5. if studentNum studentmiddle then set
found to yes 6. if studentNum lt
studentmiddle then chop off the last half of
the student list 7. If studentNum gt
studentmiddle then chop off the first half of
the student list 8. if found no then print
no such student else ltstudentNum found at
array index middlegt
17
The binary search algorithm
assuming that the entries in student are sorted
in increasing order,
1. ask user to input studentNum to search for 2.
set beginning to 1 3. set end to n 4. set found
to no 5. while beginning lt end and found
no 6. set middle to (beginning end) / 2
round down to nearest integer 7. if
studentNum studentmiddle then set found to
yes 8. if studentNum lt studentmiddle then
set end to middle - 1 9. if studentNum gt
studentmiddle then set beginning to middle
1 10.if found no then print no such
student else ltstudentNum found at array
index middlegt
18
Time requirements for binary search
At each iteration of the loop, the algorithm cuts
the list (i.e. the list called student) in
half. In the worst case (i.e. when studentNum
is not in the list called student) how many times
will this happen?
n 16 1st iteration 16/2
8 2nd iteration 8/2 4 3rd iteration 4/2
2 4th iteration 2/2 1
the number of times a number n can be cut in half
and not go below 1 is log2 n. Said another way
log2 n m is equivalent to 2m n
In the average case and the worst case, binary
search is O(log2 n)
19
This is a major improvement
20
Sorting a list
First, an algorithm thats easy to write, but is
badly inefficient...
21
The Simple Sort Algorithm
given a list of positive numbers, unsorted1, ,
unsortedn and another list, sorted1, , sortedn,
with all values initially set to zero

• 1. set i to 1
• 2. repeat until i gt n
• set indexForSmallest to the index of the
  smallest
• positive value in unsorted
• 4. set sortedi to unsortedindexForSmallest
• 5. set unsortedindexForSmallest to 0
• 6. increment i

22
This algorithm is expensive!
23
Creating Algorithms is the Challenge of Computer
Science
Its not easy try this one The Traveling
Salesperson problem

• A salesperson wants to visit 25 cities while
  minimizing the total number of miles driven,
  visiting each city exactly once, and returning
  home again. Which route is best?

24
Simplify the Problem to get an intuition about it
Q Starting at A, whats the shortest route that
meets the requirements? A Obvious to anyone who
looks at the entire map. Not so obvious to an
algorithm that sees, at any one time, only
one city and its roads
How much time does this algorithm require?
25
All Paths from A of length 5
A B
C D
Number of paths to generate and check is 24 16.
26
Can you Improve the Algorithm?

• Prune bad routes as soon as possible. Whats a
  bad route?
• Look for good solutions, even if theyre
  sub-optimal. Whats a good solution?

27
This gets real bad, real fast!

• In general, the algorithms time requirement is
  (the number of roads inout of a city)
  number of cities
• Assuming the number of roads is only 2, the time
  requirement is 2number of cities , given by the
  powers of 2 table.
• Assuming a computer could evaluate 10 million
  routes per second, finding the best route for 25
  cities would require about 3.5 seconds. No
  problem!
• However, finding the best route for 64 cities
  would require about seconds,
  or hours!

28
Comparing the time requirements
n
work
2n
n2
order 10 50 100
1,000
35 30 25 20 15 10 5 0
log2n .0003 .0006 .0007
.001 n .0001 .005 .01
.1 n2 .01 .25
1 1.67 min 2n .1024 3570
4x1016 forget it!
years centuries
n
time requirements for algorithms of various
orders of magnitude. Time is in seconds, unless
stated otherwise
log2n
n
0 5 10 15
29
Conclusions

• Algorithms are the key to creating computing
  solutions.
• The design of an algorithm can make the
  difference between useful and impractical.
• Algorithms often have a trade-off between space
  (memory) and time.