CS4710

1
CS4710

• Algorithms

2
What is an Algorithm?

• An algorithm is a procedure to perform some task.
• General - applicable in a variety of situations
• Step-by-step - each step must be clear and
  concise
• Finite - must perform task in a finite amount of
  time
• Note Is not the same as an actual implementation

3
Common algorithmic categories

• Recursion
• Divide and conquer
• Dynamic programming
• Greedy

4
Recursion

• Definition
• An algorithm defined in terms of itself is said
  to use recursion or be recursive.
• Examples
• Factorialn! n x (n-1)!, n gt 11! 1
• Fibonacci sequencef(1) 1f(2) 1f(n)
  f(n-1) f(n-2)

5
Recursion

• Recursion is natural for some problems
• Many solutions can be expressed easily using
  recursion
• Lead often to very elegant solutions
• Extremely useful when processing a data structure
  that is recursive
• Warning!
• Can be very costly!
• Calling a function entails overhead
• Overhead can be high when function calls are
  numerous
• Some software is smart enough to optimize
  recursive code into equivalent iterative (low
  overhead) code

6
Recursive factorial algorithm

• (written in pseudo code)
• factorial(n)
• if n1
• return 1
• else
• return n factorial(n-1)

7
Recursive factorial code

• a recursive factorial routine in Perl
• sub factorial_recursive
• my (n) shift
• return 1 if n 1
• return n factorial_recursive(n 1)

8
Non-recursive factorial code

• an iterative factorial routine in Perl
• sub factorial_iterative
• my (n) shift
• my (answer, i) (1, 2)
• for ( i lt n i)
• answer i
• return answer)

9
Divide and conquer

• Secrets of this technique
• Top-down technique
• Divide the problem into independent smaller
  problems
• Solve smaller problems
• Combine smaller results into larger result
  thereby conquering the original problem.
• Examples
• Mergesort
• Quicksort

10
Dynamic programming

• Qualities of this technique
• Useful when dividing the problem into parts
  creates interdependent sub-problems
• Bottom-up approach
• Cache intermediate results
• Prevents recalculation
• May employ memo-izing
• May dynamically figure out how calculation will
  proceed based on the data
• Examples
• Matrix chain product

11
Matrix chain product

• Want to multiply matrices A x B x C x D x E
• We could parenthesize many ways(A x (B x (C x (D
  x E))))((((A x B) x C) x D) x E)
• Each different way presents different number of
  multiplies!
• How do we figure out the wise approach?

12
Dynamic programming applied to matrix chain
product

• Original matrix chain product A x B x C x D x E
  (ABCDE for short)
• Calculate in advance the cost (multiplies)AB,
  BC, CD, DE
• Use those to find the cheapest way to formABC,
  BCD, CDE
• From that derive best way to formABCDE

13
Greedy algorithms

• Qualities of this technique
• Naïve approach
• Little to no look ahead
• Fast to code and run
• Often gives sub-optimal results
• For some problems, may be optimal
• Examples where optimal
• MST of a graph

14
Data structures do matter

• Although algorithms are meant to be general,
• One must choose wisely the representation for
  data, and/or
• Pay close attention to the data structure already
  employed, and/or
• Occasionally transfer the data to another data
  structure for better processing.
• Algorithms and data structures go hand in hand
• The steps of your algorithm
• What your chosen data structure allows easily

15
Some of Perls built-in data structures

• scalar
• number (integer or float)
• string (sequence of characters)
• reference (pointer to another data structure)
• object (data structure that is created from a
  class)
• _at_array (a sequence of scalars indexed by
  integers)
• hash (collection of scalars selected by strings
  (keys))

16
Perl arrays are powerful!

• Can dynamically grow and shrink
• Need a queue? (FIFO)
• Can use an array
• Add data with push operator (enqueue)
• Remove using shift operator (dequeue)
• Need a stack? (LIFO)
• Can use an array
• Push data with push operator (push)
• Pop data using pop operator (pop)

17
Advanced data structures

• Linked lists
• Circular linked lists
• Doubly linked lists
• Binary search trees
• Graphs
• Heaps
• Binary heaps
• Hash tables

18
Linked list

• Consists of smaller node structures
• Each node
• Stores one data item (data field)
• Stores a reference to next node (next field)
• Allows non-contiguous storage of data

19
Linked list

• Benefits
• Can insert more data (more nodes) in middle of
  list efficiently
• Can remove data from middle efficiently
• Word processors typically store text using linked
  lists
• Allows for very fast cutting and pasting.
• Cons
• Takes up more space (for the references)

20
Graphs

• Many representations
• Adjacency lists
• Adjacency matrix
• Must consider representation when processing
• Some graphs are weighted, others not
• Some are directed, others have implied
  bi-direction
• Searches
• Depth-first search
• Uses stack
• Yields a path
• Breadth-first search
• Uses a queue
• Yields a shortest path

21
Graphs

• Graph Searches
• Depth-first search
• Uses stack
• Yields a path
• Exhaustive
• Breadth-first search
• Uses a queue
• Yields a shortest path
• Exhaustive

22
Graphs

• Greedy algorithms for graphs
• Minimum spanning tree (MST)
• Prims algorithm
• Grow an MST from a single node
• Optimal solution
• Kruskals algorithm
• Keep picking cheap edges
• Optional solution