A Decentralized and Self-Organizing Discovery Mechanism

1
A Decentralized and Self-Organizing Discovery
Mechanism

• AINS Symposium 2002
• Michael Moore (mikemo_at_ics.uci.edu)
• Tatsuya Suda (suda_at_ics.uci.edu)
• University of California Irvine
• Department of Information and Computer Science

2
Overview

• Discovery Description
• Relationship Network
• Discovery Forwarding
• Discovery Result Returning
• Simulations
• Discussion / Future Work

3
Distributed Discovery

• Discovery Query Forwarding
• Discovery Results

Query
Node
Results
4
Key Features of Assumed Context

• Distributed
• Resources, services, users
• Dynamic
• Objects often join and leave
• Locations may change in wireless
• Content changes over time
• Localized information
• Each object has a very limited view of the network

5
Discovery Examples

• E.g. Gnutella
• Users search for files
• E.g. Chat service
• Users search for currently active users
• E.g. Sensor Networks
• Each sensor contains sensor readings

6
Our Discovery Overview

• This paper
• Agents represent the resources/services/users
• Discovery used to locate agents with certain
  keywords
• Relationships
• Contain information about other agents
• Similarity, History
• Used to organize relationships

7
Keyword-Based Discovery

• Discovery of agents based on keywords
• Query contains multiple keywords describing a
  target agent
• Query results should include agents with keywords
  that best match the query

8
Definition of Relationships

• Relationships
• Contain information that the agent knows about
  other Agents. Such as
• Keywords of the agent used to calculate
  similarity
• Similarity number matching keywords / agent
  keywords

Similarity 0.33
Agent 2
beatles rock
Similarity 0.33
Agent 1
Agent 3
music player
music classic rock
Similarity 0
Agent 4
beatles fan
9

• History of relationship
• History success rate of relationshipin
  satisfying query

History 0.2
Agent 2
History 0.7
Agent 1
Agent 3
History 0.1
Agent 4
10
Relationship Organization

• Organization by similarity may cause strong
  clustering ? greater distance / partitioning of
  clusters
• Small-world clustering
• Many relationships within clusters
• A few relationships to random clusters
• ? Helps maintain connectivity among clusters

11
Establishing Relationships

• Agents attempt to acquire
• Many relationships to agents with many similar
  keywords
• A few relationships to random agents
• Agents remove relationships to
• Non-similar Agents
• Poorly performing/unused Agents (based on
  history)

12
Using Relationships in Discovery Forwarding

• Forward a query
• With greater priority to relationships that have
  keywords more similar to the query
• Break ties of equal similarity with history
• Greater priority to relationships with stronger
  history
• (Similarity used for large scale
  structure,history used for optimization)

13
Returning Discovery Results

• Results passed back along the path used during
  forwarding. Allows
• Accumulation of results from branches of search
• Update of history values used in relationship
  selection/query forwarding
• Anonymity

14
Simulations

• Emergent clustering properties
• Discovery performance
• Similarity without History

15
Emergent Clustering Properties

• Relationship selection
• Acquiring random relationships
• Removing of relationships based on desired
  small-world structure
• Localized Policy
• Many relationships to similar agents
• Few relationships to dissimilar agents

16

• Small-world clustering over time
• All agents always have 10 relationships
• Red / Blue differentkeywords
• Lines relationshipsbetween those agents
• Relationships to all otheragents not displayed

17
Shift from Random Relationshipsto Small-World
Clustering
1
2
3
4
18

• As agents acquire small-world clustering
• Agents with same keyword become clustered
• Given that discovery has found an agent with
  keyword X, more likely to find other agents with
  keyword X
• Relationships become less random
• Given that discovery has found an agent with
  keyword X, more difficult to find other agents
  with keyword Z
• Random relationships keep clusters connected

19
Discovery PerformanceSimulations

• Discovery with similarity and without history
• Scalability of discovery with similarity and
  without history
• Configuration for above simulations
• 10000 agents
• 10 relationships per agent
• 5-10 keywords per agent, 680 global keywords
• Depth-first forwarding

20
SimulationsSimilarity without History

• Relationship Selection, Query forwarding based
  only on keyword-similarity
• No history
• Query generation
• Query issued from random agent
• Query contains 4-5 keywords
• Queries ensured satisfiable

21

• Average similarity isbetween an agent andall
  its relationships
• As similarity increases,clustering
  increases,and discoveryperformance improves

22
Scalability of Similarity without History

• Algorithm performance and resource usage scales
  linear to Agent count
• 10 relationships adequate for cluster
  connectivity in considered range

23
Scalability Analysis

• Finding keyword clusters
• At every hop, discovery encounters a random
  relationship
• Proportional to frequency at which query keywords
  appear in relationships
• Searching keyword clusters
• Linear
• As agent numbers increase, each cluster size
  grows linearly
• As long as clusters remain connected
• Clusters not well structured

24
SimulationsSimilarity With History

• Relationship Selection, query forwarding based
  only on
• keyword-similarity
• Includes history
• Query Generation
• Users biased towards specific keywords
• (Any keyword out of 32 specific keywords)

25
History Similarity

• Search Times
• No History 250 simulator cycles
• With History 110 Simulator cycles

26
History Simulations

• History definition and analysis preliminary
• Would like to show
• Potential history definitions
• How/why does history work?
• Impact on robustness
• Other capabilities

27
Remote history

• Each agent can store history about itself
• Ask another agent Are you good at discovery?
• Response I perform this well at discovering
  these things or maybeGiven what resources I
  have access to, I can do well in general
• Allows accumulation of history across many
  different queries from many other agents
• Useful indicator of expected performance
• Requires trust

28

• Can we trust discovery routing information passed
  from another agent?
• Routing information corrupt
• Routing information would be good if my traffic
  actually had enough priority
• Other unexpected complexity
• Next hop is actually an archenemy (compromised
  agent) who innocently always drops my discovery
  queries

29

• Each agent can store history per relationship
• This relationship has done well at discovering
  these thingsorI have passed my query along
  this relationship many times, and have failed
  many times
• Allows mechanism to determine what is real,
  emergent state of a relationship
• Ideally Agents can realize emergent relationship
  network characteristics through history

30

• These two types of histories can be viewed as
  Bayesian
• Prior probability remote history / heuristics
• Posterior probability locally accumulated
  history
• Accordingly we can define
• Likelihood of successful discovery Fprior
  probability, posterior probability
• Likelihood continually adapts based on new history

31

• With such a model
• Relationship Organization / Forwarding takes on a
  more probabilistic approach
• Adaptive to dynamic / unexpected situations

32

• Other Research Issues
• How to efficiently store / aggregate history
  information for discovery
• History requires additional state
• E.G. Minimize history overhead
• Local history stored as a reactive measure to
  unusually high failure rates

33
Future Work

• Further evaluation of dynamics
• Improvement in convergence rate of
  similarity-based network
• Integration / evaluation of history policy

34
Overview

• Discovery Description
• Relationship Network
• Discovery Forwarding
• Discovery Result Returning
• Simulations
• Discussion / Future Work