From Secretive Simple to Secretive Sophisticated Searching

1
From Secretive Simple to Secretive Sophisticated
Searching
Imperial Oil Summer Institute for Computer
Studies Teachers

• J.P. PrettiSchool of Computer ScienceWednesday,
  August 17, 2005

2
Abstract

• Neat and effective techniques for searching are
  not always well known. We often wow students used
  to performing a linear scan on arrays by
  introducing them to binary search. However, in
  some circumstances, self-organizing lists are
  better and much easier to implement. They can
  form the basis for a very accessible high school
  assignment. For the truly keen, skip lists are
  another little-known data structure offering a
  sound alternative to well-known binary search
  trees. In this talk, we will discuss both
  techniques. This will provide both concrete ideas
  for a high school computer science course and
  enrichment for teachers and students alike.

3
Themes

• very useful techniques can be relatively unknown
• good ideas can be surprising simple
• good solutions can be very complex
• cant ignore probabilities
• understanding trade-offs requires deep insight
  into issues

4
Outline

• Analysis of Linear Scan
• Ordering the Sequence
• Self-Organizing Lists
• Binary Search Trees
• Skip Lists
• Please keep me in line
• Let me know if I am boring you with things you
  know or using unfamiliar jargon and making silly
  assumptions

5
Searching

• input
• sequence of keyed items (of any type)
• target key
• output
• full item with target key if it exists
• useful for looking things up in a dictionary
• common application database querying
• not the same as pattern matching

6
Linear Scan

• start at the beginning of the sequence and
  continue until you find a matching item or you
  reach the end of the sequence
• usually simple to implement
• runtime?
• best-case
• worst-case

7
Average Case is Tricky

• 500000.5 comparisons on average if equally likely
  to search for any one of 1 million animals and we
  are always successful
• What if we are only successful ¾ of the time and
  equally likely to search for any animal in these
  cases?
• What if it just happens that we are always
  successful but we search for ladybug ½ of the
  time, zebra, ¼ of the time, cat ? of the
  time, etc ?

8
Order the Animals you Idiot!

• binary search repeatedly look at middle
  discarding the half of the list not containing
  the target
• reduces all searches to O(log n) comparisons
• also speeds up other less common operations
• shouldnt ignore the cost of keeping the list in
  order
• natural order does not always exist
• ignores probabilities

9
What is the Optimal Static Order?

• decreasing probability (i.e. most likely first
  and least likely last)
• when all probabilities are equal
• e.g. p1 ¼, p2 ¼, p3 ¼, p4 ¼
• when the probabilities differ greatly
• e.g. p1 ½, p2 ¼, p3 ?, p4 ?
• generalizing
• e.g. pi 2-i for i 1,,n-1 and pn 2-(n-1)

10
Using Optimal Order Itself Usually Isnt Practical

• dont know probabilities
• probabilities change over time
• BIG QUESTION
• Can we try to approximate optimal order?

11
Move-to-Front Heuristic

• a self-organizing list
• move an element to the front of the list every
  time it is accessed
• maintain relative order of the other elements
• slow in an array
• quick to optimal order
• sensitive to rare unexpected requests
• provably requires no more than twice the number
  of comparisons required for the optimal ordering

12
Transpose Heuristic

• another self-organizing list
• move an element one place closer to the front of
  the list every time it is accessed
• equivalent to swapping with its predecessor
• quick in an array
• slower to optimal order
• less sensitive to rare unexpected requests
• empirically better than move-to-front in practice
• GREAT BASIS FOR AN ASSIGNMENT

13
Binary Search Trees

• attempt to spread the love of O(log n) to inserts
  and deletes
• only accomplishes this on average unless
  sophisticated balancing techniques are used
• http//en.wikipedia.org/wiki/Binary_search_tree

14
Skip Lists

15
Skimming Over the Details

• http//iamwww.unibe.ch/wenger/DA/SkipList/
• http//www.geocities.com/SiliconValley/Network/185
  4/skiplist.html
• flip a coin to determine height of towers
• visit an average of 2 items per level
• O(log n) average case for search, insert and
  delete
• still O(n) worst-case but randomness is not based
  on the keys
• O(n) space
• from 1990

16
Themes Conclusion

• very useful techniques can be relatively unknown
• good ideas can be surprising simple
• good solutions can be very complex
• cant ignore probabilities
• understanding trade-offs requires deep insight
  into issues