INSTwine : A Scalable PeertoPeer Architecture for Intentional Resource Discovery

1
INS/Twine A ScalablePeer-to-Peer Architecture
for Intentional Resource Discovery

• Magdalena Balazinska, Hari Balakrishnan, and
  David Karger
• MIT Laboratory for Computer Science
• http//nms.lcs.mit.edu/

2
Problem Description

• Abundant ubiquitous computation and communication
• Increasingly large computing environments
• Dynamic environments
• Many possible cool applications
• Locate resources using intentional descriptions

3
INS Overview
Client describes attributes of required resources
Self-configuring resolver network
Resources advertise their capabilities
INR Intentional Name Resolver
4
Resource Discovery Goals

• Allow client applications to locate services and
  devices
• Handle sophisticated resource descriptions
• Handle dynamism in the operating environment
• Scale to large numbers of resources

5
Existing Solutions for Scalability
Partitioning
Cameras
Bldg 3
Resolver
Resolver
?
Floors 1-3
Floors 4-6
Bldg 1
Bldg 2
Resolver
Resolver
Resolver
Resolver
6
Existing Solutions for Scalability
Hierarchies
Resolver
Resolver
Resolver
Resolver
Resolver
Resolver
Resolver
7
INS/Twine Contributions

• Collaborating peer resolvers no content or
  location constraints on queries
• Scalability and load distribution via hash-based
  partitioning of resource descriptions among
  resolvers
• Semi-structured resource descriptions with
  arbitrary attribute-set
• Network dynamism
• Designed for an environment where all resources
  are equally important to users

8
INS/Twine Approach Overview
resource camera
resource motion sensor
Resolver
Resolver
subject traffic
Resolver
Resolver
Resolver
Resolver
Resolver
subject traffic
resource motion sensor subject traffic
resource camera subject traffic
9
INS/Twine Approach Overview

• A resource advertises its descriptions and
  network location to any resolver
• The description is converted into a canonical
  form attribute-value tree (AVTree)
• Using the content of the advertised description,
  the resolver determines which resolvers should
  know about the resource
• The resolver forwards the description to these
  resolvers for storage
• Similarly for queries

10
Architecture of One Resolver
h hash(a1v1-a2v2)
11
Splitting Descriptions into Strands
Resource description AVTrees
Six strands

• Each strand is then hashed into a 128 bit value
  which determines the nodes that will store the
  resource information.
• All queries, even short stranded queries require
  asking only one resolver!

12
Distributed Hash Table Chord
N5
N10
N110
N20
N99
Circular ID Space
N32
Stores key-values for keys 21..32
N80
N60
Keys 33..60

• Nodes and keys have 160-bit IDs
• Chord maps IDs to successor
• Successor Node with next highest ID

13
Basic Lookup
N120
N10
Where is key 80?
N105
Successor pointer
N32
N90 has K80
N90
K80
N60
14
Finger table allows log(N)-time lookups
½
¼
K log(n) immediate Successors for
robustness Stabilization methods for concurrency
1/8
1/16
1/32
1/64
1/128
fingeri points to successor (n 2i) log(n)
fingers in all
N80
15
Back to Example
resource camera
resource motion sensor
Resolver
Resolver
subject traffic
Resolver
Resolver
Resolver
Resolver
Resolver
subject traffic
resource motion sensor subject traffic
resource camera subject traffic
16
Properties of INS/Twine

• For a resource description with a attributes, t
  at the top-level
• Number of strands is s 2a t
• For R resources, S strands, K replication level,
  and N resolvers
• Storage requirement at each resolver (RSK)/N
• Advertisement
• SK resolvers contacted ( O(log N) for key
  routing)
• Query
• K resolvers contacted ( O(log N) for key
  routing)
• 100 success rate for less than K failures
• Failure rate decreases exponentially with K

17
State Management

• Resources join, move, leave and fail
• Resolvers join and fail
• How to maintain consistency while achieving fault
  tolerance?
• Hard state
• Soft state
• Hybrid solution implemented in INS/Twine

18
State Management
D
D
D
d
INR Intentional Name Resolver
19
State Management
Remove
Remove
Remove
INR Intentional Name Resolver
20
State Management
D
D
D
d
INR Intentional Name Resolver
21
State Management
Expire
Expire
Expire
INR Intentional Name Resolver
22
Evaluation Data Distribution
Data distribution rather even. Each resolvers
holds a small fraction of resource descriptions
23
Evaluation Query Resolution
Even distribution of queries among resolvers
24
Conclusion

• Intentional resource discovery
• Scalable peer-to-peer network of resolvers
• Hash-based mapping of resource descriptions to
  resolvers
• Dynamic and even distribution of resource
  information and queries
• Handles dynamism of resources and resolvers

http//nms.lcs.mit.edu/projects/twine/
25
Appendix
26
INS Overview
INR Intentional Name Resolver
27
Describing Resources

• INS name-specifier

• XML

• AVTrees

28
Problems using concatenation

• If numerous resources share the same prefix, some
  nodes may receive significantly more load than
  others
• Fully solving short stranded queries requires the
  colaboration of a linearly growing number of
  resolvers (with respect to network size)
• 1) and 2) are contradictory requirements!

29
Distributed Hash Table Chord
A distributed hash-table is used to map keys onto
resolvers efficiently
From Chord A Peer-to-Peer Lookup Service for
Internet Applications Ion Stoica, Robert Morris,
David Karger, Frans Kaashoek, Hari Balakrishnan
Proc. ACM SIGCOMM Conf., San Diego, CA,
September 2001.
30
Problems using prefixes

• More insertions for each resource. Small factor
  since we expect descriptions to be rather short
• Very popular prefixes may overload certain nodes
  many advertisements and queries gt the prefix
  should then become unusable
• Nodes stop storing resources for that prefix
• Nodes answer queries for the prefix specifying
  that they provide a partial answer due to the
  vague nature of the query