Napster. Decentralized storage of actual content ..

1
What can agents do for P2P systems

• Bin Yu
• Department of Computer Science
• North Carolina State University

2
What is P2P

• Definition
• A distributed system in which all nodes have
  identical responsibilities, and all communication
  is symmetric.
• An application-level Internet on top of the
  Internet
• Resource sharing on a massive scale
• Files, cycles, equipment, people...
• Conferences
• OReilly P2P conference 2001(conferences.oreilly.
  com/p2p/)
• First International Workshop on Peer-to-Peer
  Systems (IPTPS '02)
• (http//www.cs.rice.edu/Conferences/IPTPS02/)

3
Historical Perspective

• P2P is nothing new ARPANET
• Internet was fundamentally designed to be a P2P
  system
• any 2 computers could send packets to each other
• no firewalls / no network address translation
• no asymmetric connections (V.90, ADSL, cable,
  etc.)
• The popularity of www, telnet, ftp... changed the
  paradigm to client/server
• Server
• Service Provider
• Powerful machine to server a large number of
  clients
• Client
• Service Consumer
• client, machine with limited capacity, is used to
  request services

4
P2P today

• Many emerging applications
• Napster, Gnutella, Freenet
• P2P Properties
• no central coordination
• no peer has a global view of the system
• global behavior emerges from local interactions
• all existing data and services are accessible
  from any peer
• peers are autonomous
• peers and connections are unreliable

5
P2P File-sharing

• Napster
• Decentralized storage of actual content
• transfer content directly from one peer (client)
  to another
• Centralized index and search
• Gnutella
• Not like Napster, with decentralized indexing
• Search via flooding
• Direct download

6
Napster
128.1.2.3
(xyz.mp3, 128.1.2.3)
Central Napster server
7
Napster
128.1.2.3
xyz.mp3 ?
128.1.2.3
Central Napster server
8
Napster
128.1.2.3
xyz.mp3 ?
Central Napster server
9
Gnutella
10
Gnutella
xyz.mp3 ?
11
Gnutella
12
Gnutella
xyz.mp3
13
Challenge

• Location Resolution
• Given an object (might be name, attribute, or
  even content)
• Return a channel to a node (peer) that has that
  object
• Approaches
• Centralized Index (Napster)
• Broadcast information to be resolved (Gnutella)
• Distributed Hashing (Chord, CAN)

14
Distributed Hashing
Objects
Nodes

• 1. Map both objects and nodes into some topology
  (id space)

15
Distributed Hashing
Objects
Nodes

• 1. Map both objects and nodes into some topology
  (id space)
• 2. Each node owns some neighborhood in the
  topology, has channel to some neighbors

16
Distributed Hashing
Objects
Nodes

• 1. Map both objects and nodes into some topology
  (id space)
• 2. Each node owns some neighborhood in the
  topology, has channel to some neighbors
• 3. Topological structure lets query be routed to
  the owner of a given point

17
Chord - Basic Idea

• Topology is a ring of ordered, fixed-size IDs
  (say 32 bits)
• Node ID based on IP address, object ID based on
  name, content, ...

0
18
Chord - Basic Idea

• Nodes own the part of the ID space between
  their ID and their predecessors ID.

0
19
Chord - Basic Idea

• Each node has a channel to its successors at
  distances 1, 2, 4, 8, 16, ..., 2(m-1)
• where m log_2 of the ring size (32 in this case)

0
20
Chord Resolution

• Get ID of desired object
• Find the last node whose ID is LESS than the
  desired ID
• Look in finger table to find farthest-away
  neighbor whose ID is LESS than the desired ID
• Ask it for somebody closer
• That nodes successor is the owner of the
  object

21
CAN Basic Idea

• Topology is an N-dimensional torus
• N2 for simple examples
• Each node is responsible for a subrange in each
  dimension
• Space is partitioned among all nodes
• Route via neighbors -- move in direction of
  destination

22
CAN simple example
1
23
CAN simple example
1
2
24
CAN simple example
3
1
2
25
CAN simple example
3
1
4
2
26
CAN simple example
27
CAN simple example
(K,V)
(a,b)
retrieve (K)
insert (K,V)
hash(K) (a,b)
28
CAN routing table
29
CAN routing
(a,b)
(x,y)
30
Quick Review

• Two similar approaches to locating objects by
  computed routing
• Similar to Manhattan Street Networks
• Mainly for distributed storage systems.
• All these P2P networks ignore underlying
  topology!
• Each node has relatively simple function
• Network is not reconfigurable, and there is no
  learning happened
• Brute-force searching, and broadcast the request
  to all the peers
• Some networks, i.e., social networks, can not be
  partitioned by IP.

31
Agent-based P2P networks

• Software agents
• Computer programs which can perform a set of
  tasks autonomously.
• How to find an appropriate service or person
• Through referrals
• Approach automate the process using software
  agents through referrals.
• Agent-based referral networks
• Software agents cooperate to direct requests
  toward appropriate service or person.

32
Agent-based Referral Networks

• Referral systems
• MINDS 1987
• ReferralWeb 1996,1997
• A computational model of agent-based referral
  networks
• Each node is represented a software agent
• Learn models of each other in term of
• Expertise (ability to produce correct domain
  answers)
• Sociability (ability to produce accurate
  referrals)
• Cooperativeness (willingness to produce answers
  or referrals)
• .

32
33
Global View of Referral Networks
34
Why is the idea feasible?

• Relative short distance between any two nodes
• Small-world phenomenon six (5.5) for human
  social networks of USA (Stanley Milgram, 1960s)
• The relative small value indicates
• Intelligent software agents can follow only the
  relevant links and find the desired experts.

35
Paths to the Expert(s)
A
Mark
B
Jenny
C
User modeling
36
Paths to the Expert(s)
A
Mark
B
Jenny
C
User modeling
37
Paths to the Expert(s)
A
Mark
D
B
Jenny
C
User modeling
E
Uncooperative agents
38
Paths to the Expert(s)
A
Mark
D
B
Jenny
C
User modeling
E
Uncooperative agents
39
Paths to the Expert(s)
A
Mark
D
E
B
Jenny
C
User modeling
Uncooperative agents
Note that all of the queries were sent out from
Jenny. A referral graph encodes
how the computation spreads from
Jenny and referrals or answers are sent back to
Jenny.
40
Referral Graph
Ar
A2
A1
41
Referral Graph
Ar
A2
A1
A3
42
Referral Graph
Ar
A1
A2
A3
A4
43
Referral Graph
Ar
A1
A2
A3
A6
A4
Ar
Root of the graph
A5
Node has been visited
A1
Node has not been visited
A5
Redundant referral
44
Referral Graph
Ar
A1
A2
A3
A6
A4
A5
Question node A5 and A6, which should be visited
first?
45
Weighted Referral Graph
1.0
Ar
0.5
0.6
A1
A2
0.5
0.6
1.0
0.5
A3
A6
0.3
0.8
1.0
1.0
A4
0.3
1.0
A5
0.3
46
Credits/Penalties Propagation
1.0
Ar
0.5
0.6
A1
A2
0.5
0.6
1.0
0.5
A3
A6
0.3
0.8
Answer
1.0
1.0
A4
0.3
1.0
A5
0.3
Suppose A6 returns an answer, then Ar will update
the expertise for A6 and sociability for A1, A2,
A3, A4.
47
Research Challenges

• Improving the accuracy of referrals
• User modeling and multiagent learning
• Avoiding interaction with undesirable
  participants
• How to judge the trustworthiness of one agents
• Studying key properties of referral networks
• Evolution of referral networks
• Transition to small-world networks through
  interactions.
• Protocols that foster the small-world phenomenon.

47
48
Vector Space Model

• Let D d1, d2, , dn denotes a collection of
  documents. t1, t2, tp be the dictionary (a
  set of all the words)
• Each document d is represented as a p-dimensional
  vector d ?dt1, dt2, dtp?
• where tfi is the number of times word ti appears
  in document d (the term frequency),
• dfi is the number of documents in the collection
  which contain ti (the document frequency),
• n is the number of documents in the collection,
• Tfmax is the maximum term frequency over all
  words in D.

49
Ontology

• Understand the information context
• Ontology is a set of definitions of formal
  vocabulary
• Class hierarchy of AI domain.
• We manually construct an ontology for AI domain
• Totally 19 domains
• AI architecture
• Agents and multiagent systems
• Planning and search
• Vision and robotics

50
User Modeling
Expertise as a term vector EPi, ?LC1, LC2,
LCp?

• Each agent learns models of others based on
  experience
• When a good service is obtained, the expertise of
  the provider is revised upwards as is the
  sociability of those who gave referrals to it.
• When a poor service is obtained, the revisions
  are downward.

51
When to answer a query?

• Definition 1
• Given a query vector Q ?q1, , qn? and an
  expertise vector E ?e1, , en?, the similarity
  between Q and E is defined as
• Rule 1
• Given a query vector Q with the expertise domain
  Ci and a threshold ?canAnswer, where 0 ?
  ?canAnswer ?1, it says there is a good match
  between the user Pi and the query Q for a domain
  Ci if

52
When to generate referrals?

• Definition 2
• Given a query vector Q with the domain Ci, the
  relevance of a query Q to any neighbor Pj is
  defined as
• where EPj and SPj are the expertise and
  sociability of agent Pj, respectively and ? and
  (1- ?) are the weights given to sociability and
  expertise.
• Rule 2
• Given a query vector Q with the domain Ci, and a
  threshold ?canReferral, a neighbor is relevant to
  Q if
• 
  for a value of ?.

53
Architecture
Jenny
Mark
Profile
NeighborModels Cache
Agent
Referral Networks
MARS is composed of a registration server and a
bunch of software agents.
54

Feedback Queries
Answers


Queries
Answers Referrals

Queries/Answers/Referr
als Outgoing messages
Incoming
messages
GUI
Learner
Collection-P
Profile
Matchmaker
Planner
Collection-N
NeighborModels
Close Friend List
Wrapper
Classifier
Heuristics
Priority queue
55
Control Flow
GUI
Communication
4. Update
1. Incoming Messages
5. Get new message (Queries/answers)
New Message Notifier
Incoming Message Processor (Update every 5
minutes)
6. Send message
2. Create message queue
2. Create message queue
New Message Viewer
Send Queries
8. Outgoing Messages
Out-going Message Processor
3. Send out message
Planner
Answer/Refer Evaluation
Learner
7. User feedback
. . . .
Referral Graph Builder
NeighborModels Profile
56
Main windows of MARS
57
Sign up window of MARS
58
Window for incoming queries/answers
59
Conclusion

• A computation model of agent-based referral
  networks
• Improving the accuracy of referrals
• A natural way for people to seek information
• Applied in building multiagent systems in general
• A probabilistic model of distributed reputation
  management
• Leads to a decentralized society in which agents
  help each other weed out undesirable players.
• A prototype system MARS
• Limited in AI domain
• Learning the knowledge in general is nontrivial.
• Privacy when sharing with users email account.

60
Future Work

• Evaluate the efficiency of referral networks
• Reconstruct the social networks using AAAI
  (1980-2000) and IJCAI (1981-1999) proceedings
• Visualize the whole network using KrackPlot
  and/or UNINET
• Economic model of referral networks
• Incentive of help
• Payment systems
• Trust model of referral networks
• Lying and rumors

61
Wants to know more?

• Book
• Peer-to-peer harnessing the power of disruptive
  techniques, Andy Oram (ed.) OReilly
  Associates, Inc.
• Conference
• OReilly P2P conference 2001(http//conferences.o
  reilly.com/p2p/)
• First International Workshop on Peer-to-Peer
  Systems (IPTPS '02)
• (http//www.cs.rice.edu/Conferences/IPTPS02/)
• AAMAS-02 Workshop on Regulated Agent-based Social
  systems
• http//www.informatik.uni-hamburg.de/TGI/events/ra
  sta02/

62
Bibliography

• Journal Papers
• Bin Yu and Munindar P. Singh, Distributed
  Reputation Management for Electronic Commerce,
  Computational Intelligence, 2002, to appear
• Bin Yu, Mahadevan Venkatraman and Munindar P.
  Singh, An Adaptive Social Network for Information
  Access Theoretical and Experimental Results,
  Journal of Applied Artificial Intelligence, 2002,
  to appear.
• Munindar P. Singh, Bin Yu and Mahadevan
  Venkatraman, Beyond Communication Linking People
  and their Communities, Communications of the ACM,
  2001,44(4)49-54
• Conference Papers
• Bin Yu and Munindar P. Singh, An Evidential Model
  of Distributed Reputation Management, In
  Proceedings of First Joint Conference on
  Autonomous Agents and Multiagent Systems, 2002,
  to appear
• Bin Yu and Munindar P. Singh, Emergence of
  Agent-based Referral Networks (poster), In
  Proceedings of First Joint Conference on
  Autonomous Agents and Multiagent Systems, 2002,
  to appear

63

• Bin Yu and Munindar P. Singh, Towards a
  probabilistic model of distributed reputation
  management, Proceedings of Fourth International
  Workshop on Deception, Fraud and Trust in Agent
  Societies, pages 125-137, 2001
• Bin Yu and Munindar P. Singh, A Social Mechanism
  of Reputation Management in Electronic
  Communities, Proceedings of Fourth International
  Workshop on Cooperative Information Agents, pages
  154-165, 2000.
• Mahadevan Venkatraman, Bin Yu and Munindar P.
  Singh, Trust and Reputation Management in a
  Small-World Network, accepted by ICMAS'2000
  (Poster), Proceedings of Fourth International
  Conference on MultiAgent Systems, pages 449-450.
• Bin Yu, Mahadevan Venkatraman and Munindar P.
  Singh, A Multiagent Referral System for Expertise
  Location, Proceedings of AAAI99 Workshop on
  Intelligent Information Systems, pages 66-69,
  1999.
• http//www4.ncsu.edu/byu

64
Acknowledgements

• Dr. Munindar P. Singh (my thesis advisor)
• For his advice and support.
• Dr. Henry A. Kautz
• For his encouragement and discussion.
• My other committee members
• Dr. James Lester, Dr. Carla Savage, especially
  Dr. Peter Wurman for their time and valuable
  comments.
• People working on the MARS project
• Mahadevan Venkatraman, Amit Chopra
• Wentao Mo, Paul Jose Palathingal

65
The End