Algorithms and Data Structures

1
Algorithms and Data Structures

• Objectives To review the fundamental algorithms
  and data structures that are commonly used in
  programs. To see how to use and implement these
  algorithms and data structures in different
  languages and to see what language and library
  support exists for them.

2
Topics

• Arrays and Vectors
• Lists
• Linear/Binary Search
• Quicksort
• Hash Tables - Dictionaries
• C, C, Java, Perl

3
Arrays

• Sequence of items
• Indexable (can refer to any element immediately,
  by its index number)
• (Conceptually) contiguous chunks of memory

4
Arrays

• Access constant-time (?(1))
• Searching
• Sorted array - ?(lg(n))
• Unsorted - ?(n)
• Inserting/removing
• Unordered - ?(1)
• (Add to end, or swap last guy w/deleted guy)
• Ordered - ?(n)
• Need to make (or fill in) a hole, move n/2
  elements (on average) to maintain relative order

5
Resizing Arrays

• If the number of elements in an array is not
  known ahead of time it may be necessary to resize
  the array.
• Involves dynamic memory allocation and copying
• To minimize the cost it is best to resize in
  chunks (some factor of its current size)

6
Libraries - Arrays

• Arrays (lists) in Perl, Bash, Python, Awk, resize
  automatically, as do C vectors, and the Array
  in Java
• This does not mean that it is free. Depending on
  the task, what goes on underneath the hood may be
  important
• We shall create our own machinery in C

7
Some C memory things

• void malloc(int n) allocates n contiguous
  bytes from heap, returns address of first byte
  (NULL if it failed)
• free( void p ) returns memory addressed by p
  to the heap (does nothing to p itself)
• void memmove( void d, void s, size_t n )
  moves n bytes from s to (possibly overlaping
  region at) d
• void memcpy( void d, void s, size_t n )
  copies n bytes from s to (non-overlaping region
  at) d
• sizeof() actually an operator, returns size, in
  bytes, of given object or type
• void realloc( void src, size_t n ) attempts
  to resize array for you, and copy the stuff over.
  Returns ptr to new location (might be the same),
  NULL if it fails.

8
Growing Arrays in C

• enum INIT_SIZE1, GROW_FACTOR2
• int curr_size INIT_SIZE
• int nr_elems 0 / of useful elements /
• int a (int)malloc( INIT_SIZE sizeof( int
  ))
• / some stuff here /
• / attempt to insert 24 /
• if( nr_elems gt curr_size ) / need to grow /
• int t realloc( a, curr_sizeGROW_FACTORsizeo
  f( int ))
• if( t ! NULL ) / success! /
• curr_size GROW_FACTOR
• a t
• anr_elems 24
• else
• / FAILURE! /

9
Lists

• A sequence of elements
• Not indexable (immediately)
• Space is allocate for each new element and
  consecutive elements are linked together with a
  pointer.
• Note, though, that the middle can be modified in
  constant time.

head
NULL
data 1
data 2
data 3
data 4
10
Lists

• Access (same as searching) linear, ?(n)
• Modifying anywhere constant time, ?(1)
• ?(1) time to insert at front, ?(n) to append
  unless pointer to last element kept

11
Lists in Python

• The list-type in Python is really an array (I
  think. Somebody experiment? How would you?).
• We can make a linked-list in a very LISP way
  build it out of simple pairs
• 1st element the data at that node
• 2nd element the rest of the list

12
Lists in Python

• L
• add 24 to front
• L 24, L
• print L
• add 3 to the front
• L 3, L
• print L
• Would output
• 24,
• 3, 24,

13
Lists in Python append to end

• append the number 86
• t L start at beginning
• get hands on last cell
• while t !
• t t1 move to next "cell", or node
• t.extend( 86, )

14
Lists in Python searching

• def search( L, t )
• """Return cell of L that contains t,
• None if not found"""
• while L !
• if L0 t
• return L
• L L1 move to next "cell"
• return None didn't find it

15
Apply function to elements

• def apply( l, fn )
• while l !
• fn( l )
• l l1
• fn is any function that takes a single cell, does
  something

16
Lists in Python apply

• Print list
• def printCell( cell )
• print cell0,
• apply( L, printCell )
• Add 2 to each element in list
• def add2( cell )
• cell0 2
• apply( L, add2 )

17
Lists in C

• typedef struct sNode sNode
• struct sNode / a node (cell) in a
  singly-link list /
• int data / the payload /
• sNode next / next node in list /
• / newitem create new item from name and value
  /
• sNode newNode( int d )
• sNode newp
• newp (sNode) malloc( sizeof( sNode ))
• if( newp ! NULL )
• newp-gtdata d
• newp-gtnext NULL
• return newp

18
Lists in C

• / addfront add newp to front of listp
• return ptr to new list /
• sNode addfront( sNode listp, sNode newp )
• newp-gtnextlistp
• return newp
• list addfront( list, newitem( 13 ))

19
Append Element to Back of List

• / append add newp to end of listp
• return ptr to new list /
• sNode addend( sNode listp, sNode newp )
• sNode p
• if( listp NULL )
• return newp
• for( plistp p-gtnext!NULL pp-gtnext )
• p-gtnext newp
• return listp

20
Lookup Element in List

• / lookup linear search for t in listp
• return ptr to node containing t, or NULL /
• sNode lookup( sNode listp, int t )
• for( listp ! NULL listp listp-gtnext )
• if( listp-gtdata t )
• return listp
• return NULL / no match /

21
Apply Function to Elements in List

• / apply execute fn for each element of listp
  /
• void apply( sNode listp, void (fn)(sNode ),
• void arg )
• for ( listp ! NULL listp listp-gtnext)
• (fn)( listp, arg ) / call the function /
• void (fn)( sNode ) is a pointer to a void
  function of one argument the first argument is
  a pointer to a sNode and the second is a generic
  pointer.

22
Print Elements of a List

• / printnv print name and value using format in
  arg /
• void printnv( sNode p, void arg )
• char fmt
• fmt (char) arg
• printf( fmt, p-gtdata )
• apply( list, printnv, d )

23
Count Elements of a List

• / inccounter increment counter arg /
• void incCounter( sNode p, void arg )
• int ip
• / p is unused we were called,
• there's a node /
• ip (int ) arg
• (ip)
• int n 0
• / n is an int, n is its address (a ptr) /
• apply( list, inccounter, n )
• printf( d elements in list\n, n )

24
Free Elements in List

• / freeall free all elements of listp /
• void freeall( sNode listp )
• sNode next
• for ( listp ! NULL listp next)
• next listp-gtnext
• free(listp)
• ? for ( listp ! NULL listp listp-gtnext)
• ? free( listp )

25
Delete Element in List

• / delitem delete first t from listp /
• sNode delitem( sNode listp, int t )
• sNode p, prev NULL
• for( plistp p!NULL pp-gtnext )
• if( p-gtdata t )
• if( prev NULL ) / front of list /
• listp p-gtnext
• else
• prev-gtnext p-gtnext
• free(p)
• break
• prev p
• return listp

26
Linear Search

• Just exhaustively examine each element
• Works on any list (sorted or unsorted)
• Only search for a linked-list
• Examine each element in the collection, until you
  find what you seek, or you've examined every
  element
• Note that order of examination doesn't matter
• Need ?(n), worst and average

27
Linear Search - example

• L range( 1, 11 ) fill w/numbers 1-10
• random.shuffle( L ) mix it up
• target 7 we're looking for 7
• i 0
• while i lt len( L ) and not bFound
• if Li target
• break
• i 1
• if i lt len( L ) we found it!
• print str(target) \
• " was found at index " str(i)

28
Binary Search

• Only works on sorted collections
• Only works on indexed collections (arrays,
  vectors)
• Start in the middle
• Find it?
• Less than? Look in lower ½
• Greater than? Look in upper ½
• Cut search space in ½
• Need ?(lg(n)) time, worst (and avg.)

29
Binary Search

• from previous example
• L.sort() put into order
• t 7 our target
• i -1
• l 0 h len(L)-1
• while llth and i-1
• m (lh)/2 find middle element
• if( Lm t ) found
• i m
• elif t lt Lm look in left ½
• h m 1
• else look in right ½
• l m 1
• if i ! -1
• print "Found " str(t) " at " str(i)

30
Quicksort

• pick one element of the array (pivot)
• partition the other elements into two groups
• those less than the pivot
• those that are greater than or equal to the pivot
• Pivot is now in the right place
• recursively sort each (strictly smaller) group

31
Quicksort - runtime

• Best (and average) case
• Single partition, n comparisons
• Tree will have lg(n) levels
• Runtime n lg(n)
• Worst
• Bad partitions
• We end up with n levels
• Runtime n2

32
Partition
unexamined
p
last
i
e
unexamined
lt p
p
gt p
b
b1
last
i
e
lt p
p
gt p
b
last
e
33
Quicksort the recursive algorithm

• def qsort( L, b, e )
• base case
• if b gt e nothing to do - 0 or 1 element
• return
• partition returns index of pivot element
• piv partition( L, b, e )
• Note that the pivot is right where it belongs
• It does not get sent out in either call
• qsort( L, b, piv-1 ) elements in b, piv )
• qsort( L, piv1, e ) elements in ( piv, e

34
Quicksort the Partition

• def partition( L, b, e )
• "Return index of pivot"
• move pivot element to L0
• swap( L, b, random.randint( b, e ))
• last b
• Move ltp to left side, marked by last'
• for i in range( b1, e1 )
• if Li lt Lb
• last 1
• swap( L, last, i )
• swap( L, b, last ) restore pivot
• return last

35
Quicksort - Swap

• def swap( l, i, j )
• "swaps elements at indices i, j in list l"
• tmp li
• li lj
• lj tmp

36
Libraries - sort

• C qsort (in stdlib.h)
• C sort (algorithm library from STL)
• Java
• java.util.collections.sort
• Perl sort
• Python list.sort

37
qsort standard C library

• qsort( void a, int n, int s,
• int(cmp)(void a, void b) )
• Sorts array a, which has n elements
• Each element is s bytes
• cmp is a function that you must provide
• compares 2 single elements, a b
• qsort must pass void pointers (addresses), since
  it doesn't know the type
• cmp does, since you provide it for a given sort
• returns integer -1 if altb, 0 if ab, and 1 if
  agtb
• For sorting in descending order, return 1 if a lt b

38
qsort (int)

• int arrN
• qsort(arr, N, sizeof(arr0), icmp)
• / icmp integer compare of p1 and p2 /
• int icmp( const void p1, const void p2 )
• int v1, v2
• v1 ((int) p1)
• v2 ((int) p2)
• if( v1 lt v2 )
• return 1
• else if( v1 v2 )
• return 0
• else
• return 1

39
qsort (strings)

• char strN
• qsort(str, N, sizeof(str0), scmp)
• / scmp string compare of p1 and p2 /
• / p1 is a ptr to a string, or char, so is a /
• / ptr to a ptr, or a char /
• int scmp(const void p1, const void p2)
• char v1, v2
• v1 ((char) p1)
• v2 ((char) p2)
• return strcmp(v1, v2)

40
Dictionary (Map)

• Allows us to associate satellite data w/a key
• e.g., phone book, student record (given a student
  ID as key), error string (given an error as
  key), etc.
• Operations
• insert
• find
• remove

41
Dictionary

• The key-value pairs may be stored in any
  aggregate type (list, struct, etc)
• Using an unordered list
• constant-time insertion
• linear lookup
• Ordered list
• linear insertion
• lg(n) lookup

42
Other dictionaries

• Binary Search Tree
• lg(n) insertion, lookup (if balanced)
• Hash table
• constant-time insertion, lookup (if done well)

43
Trees

• A binary tree is either NULL or contains a node
  with the key/value pair, and a left and right
  child which are themselves trees.
• In a binary search tree (BST) the keys in the
  left child are smaller than the keys at the node
  and the values in the right child are greater
  than the value at the node. (Recursively true
  through all subtrees.)
• O(log n) expected search and insertion time
• in-order traversal provides elements in sorted
  order.

44
Examples

• In the following examples each node stores the
  key/value pair
• name a string
• value a hex , that represents the character
  (MBCS)
• As well as a reference (ptr) to each of the 2
  subtrees

45
Trees in Python

• We can use a list of 3 elements
• 0. The key/value pair
• 1. The left subtree
• 2. The right subtree
• The following is a tree w/one node (empty
  subtrees)
• T "Smiley", 0x263A, ,

46
BST Lookup - Python

• def lookup( T, name )
• """lookup look up name in tree T, return the
• cell, None if not found"""
• if T T is the empty tree
• return None
• if T00 name
• return T
• elif name lt T00 look in left subtree
• return lookup( T1, name )
• else look in right subtree
• return lookup( T2, name )

47
BST in C

• We will use a struct to hold the key, value, and
  pointers to the subtrees

48
Binary Search Tree
smiley
smiley

• typedef struct bNode bNode
• struct Node
• char name
• int value
• sNode left
• sNode right

0x263A
0x263A
smiley
zeta
smiley
Aacute
0x263A
0x03b6
0x263A
0x00c1
NULL
NULL
smiley
Acirc
smiley
AElig
0x263A
0x00c2
0x263A
0x00c6
NULL
NULL
NULL
NULL
49
Binary Search Tree Lookup

• / lookup look up name in tree treep /
• sNode lookup( sNode treep, char name )
• int cmp
• if( treep NULL )
• return NULL
• cmp strcmp(name, tree-gtname)
• if( cmp 0 )
• return treep
• else if( cmp lt 0 )
• return lookup( treep-gtleft, name )
• else
• return lookup( treep-gtright, name )

50
Hash Tables

• Provides key lookup and insertion with constant
  expected cost
• Used to create lookup table (with m slots), where
  it is not usually possible to reserve a slot for
  each possible element
• hash function maps key to index (should evenly
  distribute keys)
• H(k) -gt 0, m-1
• duplicates stored in a chain (list) other
  mechanisms are possible.

51
Dictionaries in Python

• Python (as well as Perl and Awk) has a built-in
  dictionary (associative array)
• Type is dict
• d empty dictionary
• initialise
• nvList 'Smiley' 0x263A, 'Aacute' 0x00c1
• print nvList 'Smiley'

52
Dictionaries - Python

• Add an entry
• nvList 'Zeta', 0x03b6
• Test for existence
• if nvList.has_key( 'AElig' )
• Remove an entry
• nvList.pop( 'Smiley', None )
• Iterate over keys
• for i in nvList.keys()

53
Hash Table - C

• typedef struct sNode sNode
• struct sNode
• char name
• int value
• sNode next / in chain /
• sNode symtabNHASH
• / a symbol table /

NULL
NULL
name 1
NULL
data 1
NULL
NULL
NULL
NULL
name 2
name 3
data 2
data 3
54
Hash Table Lookup

• / lookup find name in symtab, with optional
  create /
• sNode lookup(char name, int create, int value)
• int h
• sNode sym
• h hash(name)
• for (sym symtabh sym ! NULL sym
  sym-gtnext)
• if (strcmp(name, sym-gtname) 0)
• return sym
• if (create)
• sym (sNode ) emalloc(sizeof(sNode))
• sym-gtname name / assumed allocated
  elsewhere /
• sym-gtvalue value
• sym-gtnext symtabh
• symtabh sym
• return sym

55
Hash Function

• enum MULTIPLIER 31
• / hash compute hash value of string /
• unsigned int hash(char str)
• unsigned int h
• unsigned char p
• h 0
• for (p (unsigned char ) str p ! \0
  p)
• h MULTIPLIER h p
• return h NHASH