IOIACM Training Session 3

1
IOI/ACM Training Session 3

• Min-Yen KAN
• These notes available at http//www.comp.nus.edu.
  sg/kanmy/talks/

2
Topics to be covered today

• Computer algebra
• Large-number computations
• Miscellaneous sorting
• Invariants and Modular arithmetic
• Factorization and Prime numbers
• Dijkstras algorithm
• (if time permits)

3
Computer Algebra Part I

• Used to manipulate mathematical formulae in
  symbolic way
• Common operations
• Add
• Simplify
• Expand
• These notes mostly from http//www.math.wpi.edu/IQ
  P/BVCalcHist/calc5.html.

4
Data structures

• How to represent x5 2x 1?
• Dense (array) 15 04 03 02 21 -10
• Sparse (linked list) 1,52,1-1,0
• What about arbitrary expressions?
• Solution use an expression tree(e.g. abc)

5
Simplification Introduction

• But CA systems have to deal with equivalency
  between different forms
• (x-2) (x2)
• X2 4
• So an idea is to push all equations to a
  canonical form.
• Like Maple simplify
• Question how do we implement simplify?

6
Simplify - Transforming Negatives

• Why?
• Kill off subtraction, negation
• Addition is commutative

7
Simplify Leveling Operators

• Combine like binary trees to n-ary trees

8
Simplify Rational Expressions

• Expressions with and / will be rewritten so
  that there is a single / node at the top, with
  only operators below

9
Simplify Collecting Like Terms
10
Simplify Final Notes

• How about making a b and b a equivalent?
• Canonical Ordering
• First by their node type.
• variables, functions, and constants
• Then, lexicographically.
• Simplification may take more than one run through
• Iterate until no changes.

11
Large number computation Part II

• Sample problems
• Sum, difference and product of big numbers
• Division by a small number
• You can find more information on the web about
  this topic under multiprecision or arbitrary
  precision computation.
• These notes cobbled together from
  http//numbers.computation.free.fr/Constants/const
  ants.html

12
Representation

• Same way as done in computer algebra
• X x0 x1 B x2 B2 xn Bn
• where B is
• usually 10 (for convenience)
• Or 2 (for computers convenience)
• To think about what about arbitrarily long real
  numbers?

13
Canonicalization and adding

• Canonicalizing
• 12E2 34E0 gt 1E3 2E2 3E1 4E0
• Strip coefficient of values larger than B and
  carry to next value
• Reorder sparse representation if necessary
• Adding
• Iteratively add all Xi Yi together and then do a
  canonicalization.

14
Multiplying

• Given two big integers X and Y in canonical form
  
• the big integer Z XY can be obtained thanks to
  the formula
• Notes
• This is the obvious way we were taught in primary
  school, the complexity is T(N2)
• Canonicalize after each step to avoid overflow in
  coefficients
• To make this operation valid, B2 must fit in the
  coefficient

15
Karatsuba Multiplication

• We can split two big numbers in half
• X X0 B X1 and Y Y0 B Y1
• Then the product XY is given by
• (X0 BX1) (Y0 BY1)
• This results in three terms
• X0Y0 B (X0Y1 X1Y0) B2(X1Y1)
• Look at the middle term. We can get the middle
  term almost for free by noting that
• (X0 X1) (Y0 Y1) X0Y0 X0Y1 X1Y0 X1Y1

X0Y0
X0Y1 X1Y0
X1Y1
(X0 X1) (Y0 Y1)
- -
16
Karatsuba Demonstration

• 12 34
• Given X1 1, X0 2, Y1 3, Y0 4
• Calculate X0Y0 8, X1Y1 3 (X0X1)(Y0Y1)
  37 21
• Final middle term 21 8 3 10
• Solution 8 10 101 3 102 408
• Notes
• Recursive, complexity is about T(n1.5)
• Theres a better way FFT based
  multiplicationnot taught here that is T (n log
  n)
• http//numbers.computation.free.fr/Constants/Algor
  ithms/fft.html,

17
Miscellaneous Sorting Part III

• Most sorting relies on comparisons between two
  items.
• Proven to be T(n log n)
• Well go over
• Radix sort
• Counting sort

18
What is radix?

• Radix is the same as base. In decimal system,
  radix 10.
• For example, the following is a decimal number
  with 4 digits

1st Digit
3rd Digit
4th Digit
2nd Digit
19
Radix Sort

• Suppose we are given n d-digit decimal integers
  A0..n-1, radix sort tries to do the following
• for j d to 1
• By ignoring digit-1 up to digit-(j-1), form the
  sorted array of the numbers

20
Radix Sort (Example)
Sorted Array if we only consider digit-4
Original
Group using4th digit
Ungroup
21
Radix Sort (Example)
Sorted Array if we only consider digits-3 and 4
Original
Group using3rd digit
Ungroup
22
Radix Sort (Example)
Sorted Array if we only consider digit-2, 3, and 4
Original
Group using2nd digit
Ungroup
23
Radix Sort (Example)
Done!
Original
Group using1st digit
Ungroup
24
Counting Sort

• Works by counting the occurrences of each data
  value.
• Assumes that there are n data items in the range
  of 1..k
• The algorithm can then determine, for each input
  element, the amount of elements less than it.
• For example if there are 9 elements less than
  element x, then x belongs in the 10th data
  position.
• These notes are from Cardiffs Christine Mumford
  http//www.cs.cf.ac.uk/user/C.L.Mumford/

25
Counting Sort

• The first for loop initialises C to zero.
• The second for loop increments the values in C,
  according to their frequencies in the data.
• The third for loop adds all previous values,
  making C contain a cumulative total.
• The fourth for loop writes out the sorted data
  into array B.

• countingsort(A, B, k)
• for i 1 to k do
• Ci 0
• for j 1 to length(A) do
• CAj CAj 1
• for 2 1 to k do
• Ci Ci Ci-1
• for j 1 to length(A) do BCAj Aj
• CAj CAj - 1

26
Counting Sort

• Demo from Cardiff
• http//www.cs.cf.ac.uk/user/C.L.Mumford/tristan/Co
  untingSort.html
• Whats the complexity?

27
Invariants Part IV

• Chocolate bar problem
• You are given a chocolate bar, which consists of
  r rows and c columns of small chocolate squares.
• Your task is to break the bar into small squares.
  What is the smallest number of splits to
  completely break it?

28
Variants of this puzzle?

• Breaking irregular shaped objects
• Assembling piles of objects

29
The Game of Squares and Circles

• Start with c circles and s squares.
• On each move a player selects two shapes.
• These two are replaced by one according to the
  following rule
• Identical shapes are replaced with a square.
  Different shapes are replaced with a circle.
• At the end of the game, you win if the last shape
  is a circle. Otherwise, the computer wins.

30
Whats the key?

• Parity of circles and squares is invariant
• After every move
• if circles was even, still will be even
• If circles was odd, still will be odd

31
Introducing Modular Arithmetic

• Two numbers a and b are said to be equal or
  congruent modulo N
• iff their difference is exactly divisible by N,
• i.e. abs (a-b) / N 0

32
Modulus division

• From the definition one can easily derive several
  properties. For example
• If ab mod N and cd mod N then (ac)(bd) mod N
  
• If ab mod N and cd mod N then acbd mod N
• The criterion of divisibility by 9 follows from
  the fact that for every n, 10n1 mod 9. This, in
  turn, is a straightforward consequence of 10 1
  mod 9.

33
Divisibility tricks

• Because we use base 10, there are some tricks to
  test for divisibility.
• N is divisible by 3 iff sum of its digits is
  divisible by 3 (because 10n 1 mod 3)
• N is divisible by 9 iff sum of its digits is
  divisible by 9 (because 10n 1 mod 9)
• These tests can be applied recursively

34
Sam Loyds Fifteen Puzzle

• Given a configuration, decide whether it is
  solvable or not
• Key Look for an invariant over moves

8
7
12
15
1
2
3
4
1
5
2
13
8
7
5
6
Possible?
6
3
14
9
12
10
11
9
4
10
11
15
13
14
35
Solution to Fifteen

• Look for inversion invariance
• Inversion when two tiles out of order in a row
• inverted
  not inverted
• For a given puzzle configuration, let N denote
  the sum of the total number of inversions and the
  row number of the empty square. Then (N mod 2) is
  invariant under any legal move. In other words,
  after a legal move an odd N remains odd whereas
  an even N remains even.

11
10
10
11
36
Inversions remain the same

• Note if you are asked for optimal path, then
  its a search problem

b
c
a
d
37
In general

• If you are asked to verify something, sometimes
  the search problem isnt the most efficient way
  to compute the solution.
• Look for an invariant to solve the problem

38
Prime Numbers and Factorization Part V

• Sieve of Eratosthenes
• Factorization
• Fast Exponentiation
• Advanced Topics (not covered)
• Function Approximation
• Newtons Method

39
Sieve Algorithm

• Initialization
• Keep a boolean array 1...n PRIME. We will
  change entries to COMPOSITE as we go on.
• Algorithm
• Set k1. iterate towards vn
• Find the first PRIME in the list greater than k.
  (e.g., 2.) Call it m. Change the numbers 2m, 3m,
  4m, ... to COMPOSITE.
• Set km and repeat.

40
Sieve of Eratosthenes

• Demo from Univ. of Utah (Peter Alfeld)
• http//www.math.utah.edu/alfeld/Eratosthenes.htm
  l
• Notes
• If at n in the sieve, we know that all numbers
  not crossed out under n2 are also prime.

41
More divisibility casting out

• Is 3659867394557 divisible by 7?
• Cast out sevens.
• You can take any number you see in x and replace
  it by the remainder when divided by 7.
• For instance, the first two digits in x are 36.
  You can replace these by a 1, since 36/7 5
  remainder 1. If we continue this process, it
  might look something like this
• 159867390357   (replace some of the big digits)
• 152160320350   (start at the left and replace
• 12160320350    the first few digits in each
  step)
• 5160320350
• 260320350
• 50320000
• 1320301
• Not particularly efficient, but works for every
  number. Why?

42
Fast Exponentiation

• How do you calculate x256?
• x256 ((((((((x2)2)2)2)2)2)2)
• What about x255?
• Store x, x2, x4, x8, x16, x32, x64, x128 on the
  way up.
• Complexity T(log n)

43
Shortest Path Part VI

• Unweighted Shortest Path
• Weighted Shortest Path

A
C
B
D
E
F
44
ShortestPath(s)

• Run BFS(s)
• w.level shortest distance from s
• w.parent shortest path from s

45
Positive weighted shortest path
A
5
1
3
C
B
5
1
3
1
D
E
F
4
2
46
BFS(s) does not work
u
8
5

• Must keep track of smallest distance so far
• If we found a new, shorter path, update the
  distance.

w
10
1
s
v
47
Observation 1
s
w
10
8
2
6
v
48
Observation 2
s
6
6
v
6
6
6
6
6
49
Definition

• distance(v) shortest distance so far from s to
  v
• parent(v) previous node on the shortest path so
  far from s to v
• weight(u, v) the weight of edge from u to v

50
Example
s
w
10
8
2
6
distance(w) 8 weight(v,w) 2 parent(w) v
v
51
Relax(v,w)

• d distance(v) weight(v,w)
• if distance(w) gt d then
• distance(w) d
• parent(w) v

s
w
10
8
2
6
v
52
Dijkstras algorithm
A
5
1
3
C
B
5
1
3
1
D
E
F
4
2
53
Dijkstras algorithm
0
5
1
3
8
8
5
1
3
1
8
8
8
4
2
54
Dijkstras algorithm
0
5
1
3
5
8
5
1
3
1
8
8
8
4
2
55
Dijkstras algorithm
0
5
1
3
5
8
5
1
3
1
8
8
8
4
2
56
Dijkstras algorithm
0
5
1
3
5
8
5
1
3
1
10
6
8
4
2
57
Dijkstras algorithm
0
5
1
3
5
8
5
1
3
1
10
6
8
4
2
58
Dijkstras algorithm
0
5
1
3
5
8
5
1
3
1
8
6
10
4
2
59
Dijkstras algorithm
0
5
1
3
5
8
5
1
3
1
8
6
10
4
2
60
Dijkstras algorithm
0
5
1
3
5
8
5
1
3
1
8
6
10
4
2
61
Dijkstras algorithm
0
5
1
3
5
8
5
1
3
1
8
6
10
4
2
62
Dijkstras algorithm
0
5
3
5
8
1
8
6
10
4
2
63
Dijkstras algorithm

• color all vertices light blue
• foreach vertex w
• distance(w) INFINITY
• distance(s) 0

64
Dijkstras algorithm

• while there are light blue vertices
• v light blue vertex with min distance(v)
• color v aqua
• foreach neighbour w of v
• relax(v,w)

v
6
7
3
8
9
0
5
s
65
Running time
O(V2 E)

• color all vertices light blue
• foreach vertex w
• distance(w) INFINITY
• distance(s) 0
• while there are light blue vertices
• v light blue vertex with min distance(v)
• color v aqua
• foreach neighbour w of v
• relax(v,w)

66
Using priority queue

• foreach vertex w
• distance(w) INFINITY
• distance(s) 0
• pq new PriorityQueue(V)
• while pq is not empty
• v pq.deleteMin()
• foreach neighbour w of v
• relax(v,w)

67
Initialization O(V)

• foreach vertex w
• distance(w) INFINITY
• distance(s) 0
• pq new PriorityQueue(V)

68
Main loop

• while pq is not empty
• v pq.deleteMin()
• foreach neighbour w of v
• relax(v,w)

69
Main loop
O((VE) log V)

• while pq is not empty
• v pq.deleteMin()
• foreach neighbour w of v
• d distance(v) cost(v,w)
• if distance(w) gt d then
• // distance(w) d
• pq.decreaseKey(w, d)
• parent(w) v

70
Useful Sites

• http//www.cut-the-knot.org/blue/Modulo.shtml
• http//mathforum.org/k12/mathtips/division.tips.ht
  ml