Title: An Efficient Lookup Algorithm in Large Scale GRID Environment
1An Efficient Lookup Algorithm in Large Scale
GRID Environment
2002. 7. 12.
Computer Software Laboratory, ETRI
Han Namgoong, Incheol Jeong, Dukjoo Son nghan,
jic_at_etri.re.kr
21. Background
- World of Contents
- More than 106 scattered same digital contents
- music files, web pages, selling items,
- More than 108 internet connected users
- every internet applications use lookup service
- (ex) DNS(domain name server), router table etc.
- How do you find the data efficiently?
- less than 1 sec. response time?
32. Problem
Source Robert Morris, MIT
Lookup is very important - no control of
join/leave
43. Related Works
- Centralized lookup (Napster)
Source Robert Morris, MIT
O(N) state and a single point of failure - should
register/query to central server
53. Related Works
- Flooded queries (Gnutella)
Source Robert Morris, MIT
worst case O(N) messages per lookup
63. Related Works
- Routed queries(Freenet and Chord)
Source Robert Morris, MIT
Chord emphasizes efficiency and simplicity
- Efficient O(log(N)) messages per
lookup - Scalable O(log(N)) state
per node
73. Related Works (Chord)
- Key identifier SHA-1(key)
- Node identifier SHA-1(IP address)
- Both are uniformly distributed and exist in the
same ID space
How to map key IDs to node IDs?
83. Related Works (Chord)
Lookup(my-id, key-id) - Basic n my
successor if my-id lt n lt key-id call
Lookup(id) on node n // next hop else return
my successor // done Correctness depends only
on successors
93. Related Works (Chord)
- Finger table allows log(N)-time lookups
Finger i points to successor of n2i
103. Related Works (Chord)
- finger table of 3 nodes and interval
113. Related Works (Chord)
Lookup(my-id, key-id) look in local finger
table for highest node n s.t. my-id
lt n lt key-id if n exists call Lookup(id)
on node n // next hop else return my
successor // done
O(log(n)) complexity
123. Related Works (Chord)
- What is happening if nodes fail?
Failures might cause incorrect lookup
133. Related Works (Chord)
- solution of multiple nodes failures
- build and keep successor list
- Each node knows r immediate successors
- After failure, will know first live successor
- Correct successors guarantee correct lookups
- Guarantee is with some probability
- Lookup(my-id, key-id)
- look in local finger table and successor-list
- for highest node n s.t. my-id lt n lt key-id
- if n exists
- call Lookup(id) on node n // next hop
- if call failed,
- remove n from finger table
- return Lookup(my-id, key-id)
- else return my successor // done
144. Not Covered Problems in Chord
- multiple key values for GRID environment
- available resources CPU, disk space, printer,
etc. - different attributes of available resources
- amount(CPU allowed time, disk space,..)
- start/end time of availability of resources
- .
- (read only)data availability?
- only one node fails we wait the recovery of that
node - strictly sequential access ? low performance
- network partition
- partition occurs in the right before and just
after the successor list
155. Extension of Chord
- multiple key values for GRID environment
- available resources CPU, disk space, printer,
etc. - different attributes
- amount(CPU allowed time, disk space,..)
- start/end time of availability of resources
- multiple signature keys depending on resources
types - key identifier SHA-1(multiple signature keys)
- node identifier SHA-1(IP address)
- both are uniformly distributed and exist in the
same ID space
165. Extension of Chord
- (read only)data availability?
- only one node fails we wait the recovery of that
node - strictly sequential access ? low performance
- network partition
- partition occurs in the right before and just
after the successor list
?
?
175. Extension of Chord
- solution ? make replica
- evenly distribution in the same ID space
- ? in 2 replica case expose the key to the n,
n(1/3)n, n(2/3)n
Key 5
K5
Node 110
N110
K20
K80
replica 1
K80
replica 2
Circular 7-bit ID space
N32
N90
K80
1. high rate of data availability ? 2 replica
one node failure probability 0.05 ? 2 replica
provides 99.98 availability 2. network partition
? 2 replica we get unavailability of desired
data only when more than 2/3 nodes fails
185. Extension of Chord
- penalty by this extension
- constant times of existing Chord algorithm
- look up cost(worst case) O(log(n) k)
O(log(n)) - join cost O(log(n) k) O(log(n))
- leave cost O(log(n) k) O(log(n))
- route info. in each node O(log(n) k)
O(log(n)) - ? k is the number of replica
196. Conclusion
- same performance as Chords
- Efficient O(log(N)) messages per lookup
- N is the total number of nodes
- Scalable O(log(N)) state per node
- Robust survives massive failures
- solve network partition
- provide high availability
- primarily depends on the number of replica
- not covered by our approach
- practical deployment and experiment
- use of locality and(or) pattern of data
20