MCS 101: Algorithms

1
MCS 101 Algorithms

• Instructor
• Neelima Gupta
• ngupta_at_cs.du.ac.in

2
Table of Contents

• Divide and Conquer

3

• Divide the problem into smaller sub-problems.
• Solve the sub-problems recursively
• Combine the solution of the smaller sub-problems
  to obtain the solution of the bigger problem.
• Size of subproblems must reduce
• There must be some initial conditions
• The two together ensures that the algorithm
  terminates

4
Familiar Egs of divide and conquer

• Quick sort
• Merge sort (John von Neumann, 1945)

5
Time

• If T(n) is the time to sort n numbers -
• T(n)T(n/2)T(n/2)?(n)
• 2T(n/2) cn
• ?(nlog n)
• Where ?(n ) time required to merge two sorted
  lists, each of size at most n/2

6
Multiplying 2 large integers

• Suppose for simplicity, numbers are of equal
  length Eg-
• suppose the 2 number are
• 2345 and 3854
• O(n2) time by the usual successive add algorithm.
• We can reduce this time using divide and conquer
  strategy.

7
Multiplying 2 large integers

• split the numbers into roughly 2 halves
• Here x0 45 x1 23 y0 54 y1 38

8

• Mathematically
• x x1 .10n/2 x0 (1)
• y y1 .10n/2 y0 (2)
• xyx1y1.10n(x1y0y1x0).10n/2x0y
  0 (3)
• In our eg n4
• x23102 45
• y38102 54

9

• x1y1874x1y01242x0y11710x0y02430
• Using (1),(2),(3) answer came out to be
• 874104 (2952)1022430
• adding these numbers take linear time.

10
Time Analysis

• Computing
• x1y1,(x1y0y1x0),x0y0
• T(n)4T(n/2)cn
• ?(n2)
• So, no improvement. Well use a smarter way to
  compute the middle term.

11

• (x0x1)(y0y1)x1y1(x1y0y1x0)x0y0
• Thus,
• x1y0y1x0 (x0x1)(y0y1) - x0y0 - x1y1
• Thus, only 3 multiplications suffice

12

• So T(n)is reduced to
• T(n)3T(n/2)cn
• ?(nlog23)lt?(n2)
• where 3T(n/2) is the time required to multiply
• x1y1 , (x0x1)(y0y1), x0y0
  

13
Complex Roots of Unity

• Consider xn 1
• It has n distinct complex roots as follows
• ?j,n e (2 p ij)/n , j 1 to n 1
• where i v-1
• These numbers are called nth roots of unity

14
nth roots of unity

• For example, for n 2
• x 2 1 gt x 1 , -1
• e (2 p ij)/n , j 0,1

j 1 e (2 p i)/2 epi cos p i sin p
-1
j 0 e (2 p ij)/n e0 cos 0 i sin 0
1
Thanks to Megha and Mitul
15

• These can be pictured as a set of equally spaced
  points lying on a unit circle as shown in figure
  for n 8.
• Figure by Mitul
• If ?j,2n ( j 0, 1 2n-1) are (2n)th roots
  of unity then clearly ?2j,2n ( j 0, 1 n-1)
  are nth roots of unity. For j n 2n -1, roots
  repeat.
• ? j,2n e (2 p ij)/2n e (p ij)/n
• ? j,2n2 (e (p ij)/n )2 e (2 p
  ij)/n
• It is also visible from the figure below
• Figure by Mitul

16
Discrete Fourier Transform

• DFT of a polynomial with coefficient vector
  lta0,a1,.,angt is the vector y lty0,y1,.,yngt
  where
• T(n2) time to compute DFT is trivial, each yi can
  be computed in T(n) time.
• FFT is a method to compute DFT in T(n log n)
  time, It makes use of special properties of
  complex roots of unity.

17
Fast Fourier Transform

• Instead of n, we will deal with 2n
• Break the polynomial in 2 parts
• Aeven(x) a0 a2x an-2 x (n-2)/2
• Aodd(x) a1 a3x a5x2 . an-1x(n-2)/2
• A(x) Aeven(x2) x.Aodd(x2)
• Let
• p(x) Aeven(x2) a0a2x2 a4x4. an-2xn-2
  and
• q(x) Aodd(x2) a1 a3x2 a5x4.an-1xn-2
• x.q(x) a1x a3x3 a5x5an-1xn-1

Thanks to Megha and Mitul
18
Fast Fourier Transform contd

• Thus,
• yj A(?j,2n ) Aeven(?2j,2n ) ?j,2n
  .Aodd(?2j,2n )
• (2n)th root of unity nth root of unity
• Aeven and Aodd are polynomials with n terms,
  thus they can be computed at nth roots of unity,
  recursively.

19
FFT contd..

• Thus we arrive at the following recurrence to
  compute the DFT
• T(n) 2T(n/2) O(n)
• n log n
• Thus given a polynomial, its DFT can be computed
  in O(n log n) time.

20
Vandermonde Matrix

• Computing DFT is equivalent to
• n is only a control variable and can be replaced
  by 2n
• Since DFT can be computed in O(n log n) time,
  this matrix-vector product can be computed in O(n
  log n) time.

Vandermonde matrix
21
Computing DFT-1

• TheoremFor j,k 0 2n -1, (j,k) entry of V-12n
  is ?-kj,2n /2n
• Thus, given yi s , ai s and hence the
  polynomial can be computed as follows
• As before n can be replaced by 2n. This
  matrix-vector multiplication is similar to the
  previous one and hence can be computed in O(n log
  n) time.

22
Convolution

• Let A lta0,a1,.,angt and
• B ltb0,b1,..,bngt be 2 vectors
• C A o B ltc0,c1,..,c2n-2gt (Length 2n
  -1)
• C k ? ij k aibj
• AIM to obtain convolution in (nlog(n)) time

Thanks to Megha and Mitul
23
Applications polynomial multiplication

• Suppose we have two polynomials
• A(x) a0 a1x an-1xn-1
• B(x) b0 b1x bn-1xn-1
• Then, C(x) A(x).B(x)

24
Algortihm for AoB

• Let A(x) and B(x) be two polynomials with
  coefficients from the vectors A and B
  respectively.

• Compute A and B at (2n)th roots of unity
  i.e. compute A(?j,2n ) and B(?j,2n ) , j 0,1
  2n-1 using FFT. O(n log n) time.
• Compute C(?j,2n ) A(?j,2n ) . B(?j,2n ) .
  O(n) time.
• Reconstruct C(x) using FFT-1 .O(n log n)
  time.

25
FINDING CLOSEST PAIR OF POINTS

• Submitted By
• Jewel
  Pruthi(18)
• Juhi Jain(19)

26

• Problem Given a set of points
  p1,p2,--------pn,
• our aim is to find out the
  closest pair
• of points.
• Finding Closest Pair in one-dimension
• Complexity O(nlogn)
• How?
• 
  
  
• 
  
  Thanks
  to Jewel and Juhi

27

• Sort the points.(takes O(nlogn) time)
• Find distance between every pair of consecutive
  points.
• In n comparisons,we will find the distance
  between every pair of consecutive points.
• In another n comparisons,we will find minimum of
  the distances found.
• 
  
• 
  
  Thanks
  to Jewel and Juhi

28
Finding Closest Pair in two-dimension

• Proposed algorithmSort the points p1,p2,------pn
  say on increasing order of x-coordinates.
• where pi(xi,yi)
• Distance between 2 consecutive points
• dsqrt((y2-y1)2 (x2-x1)2 )
• st
  x1x2x3------xn
• Ques Will this algorithm work?
• Thanks to Jewel and Juhi
  

29

• No
• Consider p1,p2,p3 st d(p1p2) gt d(p2p3) after
  sorting p1,p2,p3 on x-coordinates.
• Algorithm returns p2p3 but we can see that p1p3
  are closer.
• 
  
  Thanks to Jewel and
  Juhi

p2
p1
p3
30

• Note For minimum distance,the two points need
  not be consecutive on x/y-axis.
• Brute Force Approach
• Calculate distance between every possible pair of
  points.
• Time Complexity- O(n2)
• Can we improve on the time complexity?
• 
  Thanks to Jewel and Juhi

31

• Divide and Conquer Approach
• Arrange the points in increasing order of
  x-coordinates say Px increasing order of
  y-coordinates say Py.
• Divide the set of n points into 2 halves Q and R
  (breaking on middle of Px).
• Compute the closest pair in Q and in R
  recursively.
• 
  
  Thanks to Jewel and
  Juhi

Q
R
Solve recursively
Solve recursively
32

• Let (q1,q2) and (r1,r2) be the closest pairs
  obtained in Q and R respectively.
• Let dmin d(q1,q2) , d(r1,r2)
• 
  Thanks to Jewel and Juhi

Q
R
(q1,q2)
(r1,r2)
Solve recursively
Solve recursively
33

• Ques Does ? a pair of points say (s1,s2) such
  that d(s1,s2) lt d ?
• Soln Let p be the point with maximum
  x-coordinate in Q and let x be its x-coordinate.
• Draw a line through p described by the
  equation L xx
• 
  
  
  
  
• 
  
  Thanks
  to Jewel and Juhi

34

Q

• 
  
• Thanks to Jewel and Juhi

This distance is x-qx
R
d
q(qx)
p
d
r(rx)
x
L

• Now in the highlighted triangle we can see that
  the length
• of horizontal line
• (x-qx) lt hypotenuse according to Pythagoras
  theorem.
• x-qxlthypotenuseltd(q,r)lt d
• Similarly we can prove that rx-x lt d(q,r) lt d

35

• Consider square boxes each of side d/2 in this
  vertical strip.
• 
  Thanks to Jewel and Juhi
  

S
d/2
qy
d/2
Square of side d/2
ry
L
36

• Claim No box contains more than 1 point.
• Proof If the 2 farthest points in the box are
  the 2 diagonal points of the square, then
• Distance between the 2 points
• v( (d/2)2 (d/2)2 ) (d/v2)
  lt d
• 2. Both the points are within Q or both are
  within R
• This implies that we have two points within Q
  (/R) with distance lt d which is a contradiction
  to the definition of d.
  
• 
  
  
  Thanks to Jewel and Juhi

37

• Claim Between any pair of 2 points q and r in S
  with d(q,r) lt d , there can be no more than 12
  points.
• Thanks to
  Jewel and Juhi

?r
?k
No of points 12
?2
?1
38

• Proof Let Py1 Pyt be the points of S in the
  increasing order of y co-ordinates.
• If there are more than 12 points between q and r
  in the above list then there will be at least 3
  rows between the y co-ordinate of q and y
  co-ordinate of r.
• Then,
• the vertical distance between q and r is
• gt 3 (d/2) gt d
• Thus, the actual distance which is gt vertical
  distance (show through fig.) gt d
• ---Contradiction to the definition of points q
  and r.
• Thanks to Jewel and Juhi

39

• For i 1 to t
• For j i1 to i 12
• Compute d(Pyi , Pyj)
• Compute the minimum of the 12t pairs above ---
  O(n) time.

40

• THANKYOU