Memoization

1
Memoization

• Algorithms
• Baojian Hua
• huabj_at_mail.ustc.edu.cn
• April 21, 2008

2
Whats memoization?

• A systematic method for memoizing the results
  of some computation or even arbitrary operations
• To save the intermediate results
• To log (debug)
• To ensure correctness (later in this course)
• Ex graph traversal (say DFS)
• To reduce complexity
• Topic of this slide

3
Motivating Example

• / For an array of string strs, we apply some
• computation trans on each string s
• /
• String strs aa, bb, cc, aa, aa,
  
• void foo(trans-gt)
• for (String s strs)
• v trans(s)
• // however, if trans is slow, the function
• // foo will be VERY expensive

4
Motivating Example

• / A very simple (but effective) hack is to
• memoize the comp that have been finished /
• String strs aa, bb, cc, aa, aa,
  
• StringMapltString, Xgt map
• void memoize(trans-gt)
• for (String s strs)
• if (lookup(map, s)!null)
• then return lookup(map, s)
• else
• v trans(s)
• enter(map, s, v)

5
Memoizing Intermediate Computations

• / Factorial numbers /
• fac(int n)
• if (n0)
• return 1
• else return nfac(n-1)
• // Its crucial to notice that in computing
• // fac(n), we computed
• // all values from fac(0) to fac(n-1)

6
Memoizing Intermediate Computations

• / So, its easy to memoize all of them with an
• auxiliary map, say an array result /
• int result
• fac(int n)
• if (n0)
• return 1
• else
• int tmp n fac(n-1)
• resultn tmp
• return tmp
• // Its a save for computing
• for (int i0 iltn i)
• fac(n)

7
Recursion Revisited

• Divide Conquer
• Divide
• Initial problems could be divided into several
  sub-problems which are NOT overlapping.
• Conquer
• Solve these sub-problems individually
• Combine
• Find the final answer by combing sub-ones

8
Divide Conquer

• Sort
• nearly all
• Select
• Searching
• bst, red-black,

9
Overlapping Recursions

• Overlapping recursions
• with overlapping sub-problems
• Ex
• int fib (int n)
• if (n0)
• return 0
• else if (n1)
• return 1
• else
• return fib (n-1) fib (n-2)
• // though simple, its rather slow (try demo )

10
Recursion Tree

11
Complexity

• The recurrence equation
• T(n) T(n-1) T(n-2)
• The sotion is T(n) \phin,
• where \phi 1.618 (golden ratio) Exponential!
• Its elegant, but not usable in practice
• The problems is that we are re-computing many
  sub-items many times

12
Another Approach

• Use an array arrn to store fib(n)
• arr0 0
• arr1 1
• for (int i2 iltn i)
• arri arri-1 arri-2
• // which not only computes fib(n), but also fib(i)

13
Moral

• The total running time is O(n)
• We never re-compute any value
• any value (array item) is computed and assigned
  just once (after the initialization)
• This is one simplest form of bottom-up memoization

14
Memoization top-down

• int arr // with proper initialization, say -1
• int fib (int n)
• if (n0)
• return 0
• else if (n1)
• return 1
• else if(arrngt0) // fib(n) has been computed
• return arrn
• else
• int tmp fib(n-1) fib(n-2)
• arrn tmp
• return tmp

15
Memoization top-down

• int arr // with proper initialization, say -1
• int fib (int n)
• if (n0)
• return 0
• else if (n1)
• return 1
• else if(arrngt0) // fib(n) has been computed
• return arrn
• else
• int tmp fib(n-1) fib(n-2)
• arrn tmp
• return tmp
• // with a lazy flavor

16
Moral

• This is the top-down memoization
• Its more attractive for
• its more natural for problem trans
• the order of the computation take care of itself
• answers for all of the sub-problems may NOT be
  computed at all

17
Table Representation

• Above examples make use of a simple table
  representation
• linear structure int arr
• In principal, its mapping
• map X-gtY
• So, any search table rep strategies works
• arrary-based, list-based, tree-based, hash,
• Next, we explore hash-table

18
Hash-table (Map)-based

• HashTableltInteger, Integergt ht new ()
• int fib (int n)
• if (n0)
• return 0
• else if (n1)
• return 1
• else if(ht.lookup(n)) // fib(n) computed
• return ht.lookup(n)
• else
• int tmp fib(n-1) fib(n-2)
• ht.insert(n, tmp)
• return tmp

19
Advantage

• More natural for problem representation
• Uniform always a map
• Dont blurred by annoying details
• Ex array positions
• Easy to extend to more complex situa
• See next example

20
Multi-dimension Recursion

• f(m, n) 0, if mgtn
• f(m, n) 1, if m0 or mn
• f(m, n) f(m, n-1) f(m-1, n-1)
• // f maps every integer tuple (m, n) to an
• // integer f(m, n)

21
Hash-table (Map)-based

• HashTableltTupleltInteger, Integergt, Integergt ht
  new ()
• int f (int m, int n)
• if (mgtn)
• return 0
• else if (m0 mn)
• return 1
• else if(ht.lookup(Tupleltm, ngt))
• return ht.lookup(Tupleltm, ngt)
• else
• int tmp f (m, n-1) f (m-1, n-1)
• ht.insert(Tupleltm, ngt, tmp)
• return tmp