Dynamic Layer Management in Superpeer Architectures

1
Dynamic Layer Management in Superpeer
Architectures

• Li Xiao, Zhenyun Zhuang, and Yunhao Liu
• IEEE Trans. On Parallel and Distributed Systems,
  Vol. 16, No.11, November 2005
• DCC Lab.
• Saeyoung Han

2
Outline

• Introduction
• Workload model
• Design of DLM
• Performance evaluation
• Discussion on the side effects of DLM
• Conclusion
• Related works

3
Introduction

• Unstructured P2P systems
• Gnutella
• Regardless of their capacity, act equal roles and
  takes same responsibilities for all the
  operations.
• As the network size increases, the weak peer will
  seriously limit the scalability of P2P systems.
• Superpeer architectures
• KaZaA, early Morpheus, based on FastTrack
  structures
• Introduce Ultra-peers into Gnutella protocol
• Lack of effective layer management schemes

4
Introduction

• Unsolved problems
• The optimal size ratio of leaf-layer to super
  layer
• An appropriate size ratio maintenance mechanism
• What types of peers should be elected to
  superlayer?
• The main contribution
• Obtain the optimal size ratio by modeling
  superpeer systems based on existing studies and
  their observation
• Propose fully distributed dynamic layer
  management algorithm, DLM
• Demonstrate that the quality of a superpeer
  system is significantly improved under the DLM
  scheme

5
Workload Model- Importance of appropriate layer
size ratio

• Superpeer networks..
• Take advantage of peers heterogeneity
• Scale better by reducing the number of query
  paths.
• Superpeers
• process and relay the queries coming from
  leaf-peers and other superpeer neighbors.
• keeps an index of its leaf-peers shared data
• Leaf-peers
• Keep a number of connections to superpeers for
  the reliability.
• Query submit and relay
• Both superpeers and leaf-peers can submit queries
• Only superpeers can relay queries and responses

6
Workload Model- Importance of appropriate layer
size ratio

• Higher search efficiency because only superpeers
  are involved in search processes
• An appropriate layer size ratio is of great
  importance.

Ex. Preconfigured threshold superpeer gt 50KB/s
7
Workload Model

• Parameters
• n peers, ns superpeers, nl leaf-peers
• each leaf peer connects to m superpeers
• each superpeer connects to ks other superpeers
  and kl leaf-peers
• the layer size ratio
• Theorem1. On the average, each superpeer connects
  to m leaf-peers and the number of superpeers is
  given by

8
Workload Model

• Workloads definition
• Won the total traffic required to perform a
  search task in a P2P networks
• Wsp the traffic overhead on a superpeer to
  perform a search task
• as a function of ?
• Superpeers perform
• maintaining connections to the neighboring
  peers? Connection workload
• processing the queries initiated from its
  leaf-peers and itself? Query workload
• relaying the queries come from its superpeer
  neighbors? Relay workload

9
Workload Model

• Connection workload
• The traffic overhead incurred to maintain the
  connections to the neighboring peers
• proportional to the number of neighbors
• inverse proportional to the average up time of
  them
• (tl and ts are the average lifetimes of
  neighboring leaf-peers and superpeers,
  respectively)

10
Workload Model

• Query workload
• The traffic overhead incurred for a peer to
  process the queries generated by its leaf
  neighbors and itself
• proportional to the number of leaf neighbors
• proportional to the query frequency of each
  peer(f query frequency of a peer)

11
Workload Model

• Relay workload
• the traffic overhead incurred to process queries
  relayed from the superpeer neighbors
• p the number of peers queried (covered) in a
  search task

12
Workload Model

• Theorem 2. In a super peer network with each
  leaf-peer connecting m superpeers and each
  superpeer connecting kl leaf-peers, to cover p
  peers, the number of superpeers that should be
  queried has a lower bound of p / (1kl) and an
  upper bound of p / (m kl) on average.

13
Workload Model

• Theorem 3. When ps ltlt ns, the probability of a
  leaf-peer connecting to more than one covered
  superpeer is very close to zero.

14
Workload Model

• Theorem 4. To cover ps p / (1 kl) superpeers,
  the number of query messages ranges from p /
  (1kl) -1 to pks / (1 kl), regardless of search
  mechanisms.

15
Workload Model

• The optimal layer size ratio ? based on the
  weighted workload W

16
Workload Model
17
Workload Model
18
Design of DLM

• How to maintain an optimal layer size ratio in a
  highly dynamic environment?
• Which peers should be promoted or demoted when
  the system has more or less superpeers than
  optimal?

19
Design of DLM

• Two goals of DLM
• maintaining the size ratio of superpeer to
  leaf-peer
• keeping the peers with larger lifetimes and
  capacities as superpeers and the peers with
  shorter lifetimes and capacities as leaf-peers
• Two metrics
• Capacity
• The ability of a peer to process and delay
  queries and query responses - bandwidth, CPU
  speed, and storage space,
• Age
• The length of time up to now since a peer joined
  the network
• Lifetime the period of time in which the peer
  participates in the P2P network

20
Design of DLM

• DLM 1
• Step 1 Information collection
• Step 2 Maintaining appropriate layer-size ratio
• Step 3 Scaled comparisons of capacity and age
• Step 4 Promotion or demotion
• DLM 2
• DLM 3
• Applying DLM to multilayer architectures

21
Design of DLM - DLM - 1

• Step 1 Information collection
• Exchange information with their neighbors using
  messages
• The leaf neighbor number of s lnn(s)
• neigh_num_reuest leafnode d ? superpeer s
• neigh_num_response superpeer s ? leaf-node d
• The leaf-peers capacity and age
• value_request between a superpeer and a
  leaf-peer
• value_response between a superpeer and a
  leaf-peer
• Higher frequency means higher accuracy and more
  traffic overhead
• An event-driven policy
• A time interval-based policy
• Piggybacking

22
Design of DLM - DLM - 1

• Step 2 Maintaining appropriate layer-size ratio
• The extent µ of inappropriateness of current
  layer size ratio compared to the optimal layer
  size ratio ?
• For a superpeer s, lnn lnn(s)
• For a leaf-peer l, lnn the average lnn value of
  the superpeer in G(l)
• µ gt 0 ? lnn gt kl m? ? too few superpeers in the
  system
• The related set of a peer G
• For a superpeer s, G(s) is the set of its current
  neighboring leaf-peers
• For a leaf-peer l, G(l) is the set of superpeers
  connected

23
Design of DLM - DLM - 1

• Step 3 Scaled comparisons of capacity and age
• The direct comparison
• system cannot adjust the layer size ratio at all
• The scaled comparison
• For a peer d,
• Xcapa, Xage two scale parameter
• Ycapa, Yage two counting variables (relative
  value of one peer compared to the peers in the
  other layer)

for all peer di in G(d) if (capacity(di) Xcapa
gt capacity(d)) Ycapa 1 / (size of G(d)) if
(age(di) Xage gt age(d)) Yage 1 / (size of
G(d))
24
Design of DLM - DLM - 1

• Step 3 Scaled comparisons of capacity and age
• Xcapa, Xage are adjusted according to the value
  of µ.
• For a superpeer,
• if it find that the system needs more superpeers,
  
• it will decrease the possibility of its demotion
• by decreasing the two scale parameters
• For a leaf-peer,
• if it finds that more superpeers are needed,
• it will increase the promotion possibility
• by decreasing the two scale parameters

25
Design of DLM - DLM - 1

• Step 4 Promotion or demotion
• For a leaf-peer l,
• Ycapa and Yage are small enough
• ? many superpeers in G(l) have smaller metric
  values
• ? l has relatively large metric value
• ? may be promoted to super-layer
• For a superpeer s,
• Ycapa and Yage are large enough
• ? many leaf-peers in G(s) have large metric
  values
• ? s has relatively smaller metric value
• ? may be demoted to leaf-layer

26
Design of DLM - DLM - 1

• Step 4 Promotion or demotion
• Two threshold Zcapa and Zage
• For a leaf-node l,
• If Ycapa and Yage are smaller than Zcapa and
  Zage, respectively, it will be promoted to a
  superpeer.
• It keeps its current connections to other
  superpeers.

27
Design of DLM - DLM - 1

• Step 4 Promotion or demotion
• For a superpeer s,
• If Ycapa and Yage are larger than Zcapa and Zage,
  respectively, it will be demoted to a leaf-peer.
• It only keeps m of its current connections to
  other superpeers and drops the connections to
  leaf-peers.

28
Design of DLM - DLM - 1

• Step 4 Promotion or demotion
• Zcapa and Zage are also adjusted according to the
  value of µ
• When more superpeers are needed,
• superpeers will increase the thresholds to reduce
  the demotion tendencies
• leaf-peers will reduce the thresholds to increase
  the promotion tendencies
• Two factors for two goals of DLM
• The layer-size-ratio factor (µ)
• Maintaining optimal layer size ratio
• Xcapa and Xage are adjusted based on µ
• The capacity-age factor
• Keeping large-capacity and large-age peers on the
  superlayer
• By using the scaled comparison method

29
Design of DLM - DLM - 2

• Neighboring superpeers exchange information
• Three policies to exchange information among
  superpeers
• Pushing exchanging
• Each superpeer sends its calculated system
  information to superpeer neighbors whenever it
  runs DLM-1.
• Pull exchanging
• Whenever a superpeer runs DLM-1, it requests its
  superpeer neighbors to send back their system
  condition.
• Periodic exchanging
• Each superpeers sends out its estimation result
  periodically.
• The performance varies according to different
  values of T
• ( Smaller T ? more exchanges and overheads, but
  more information obtained for each peer )

30
Design of DLM - DLM - 3

• Only superpeers perform the estimation process
• When one superpeer infers that the network has
  too many superpeers, it may decide to be demoted
• When it finds that more superpeers are needed in
  the network, it will inform the most eligible
  leaf neighbor and promote it to superlayer
• Three policies to select the most eligible
  leaf-peer
• Largest-Age
• Largest-Capacity
• Weighted-Metric
• For a peer d,
• ,where the Metric Weight Parameters are

31
Design of DLM - DLM - 3

• Metric-Distribution mechanism to set ?i
• Consider a peer d in a superpeers Related-Set G
  and let us use Vr,d to denote the value of metric
  r on d.
• Normalize the metric value to 0, 1 space
• Compute the standard variance of metric r
• Letting denote the weight parameter for metric
  r, set as

32
Design of DLM - Applying DLM to Multilayer
Architectures

• The k-layer architectures
• If a peer in layer i (1 I lt k) has larger
  capacity and age value, it will be automatically
  promoted to the upper layer, say i 1.
• If a peer in layer i (1 I lt k) has smaller
  capacity and age value compared with other lower
  layer peers, it will be automatically promoted to
  the lower layer, say i - 1.
• The algorithm can make sure that the peers with
  longer lifetime and larger capability are in
  higher layer.

33
Performance Evaluation

• Simulation parameters

34
Performance Evaluation

• Workload on a superpeer Wsp on varying ?
• Connection Workload, Query Workload, Relay
  Workload, Total Workload
• Most efficient and inefficient searches (Relay
  Workload)
• Both Connection Workload and Query Workload
  increase as ? increases.
• For Relay Workload, in the minimum case, the
  amount decreases slightly when ? increases,
  however, in the maximum case, it almost says
  constant, regardless of different ? values.
• In both case, Total Workload on one superpeer
  increase as ? increases. Since, a larger ? value
  means that each superpeer represents more
  leaf-peers.

35
Performance Evaluation

• Workload on the overall network Won on varying ?
• Won decreases when ? increases
• The main part of the total workload comes from
  the relay and, compared to the Relay Workload,
  the amounts of Connection Workload and Query
  Workload are negligible.

36
Performance Evaluation

• The weighted workload with a ß 0.5 as ?
  changes
• Compute the optimal value
• The most efficient case (left) 38
• The most inefficient case (right) 51

37
Performance Evaluation

• The weighted workload with a 0.3 and ß 0.7 as
  ? changes
• The optimal values of ? are slightly larger than
  before.
• larger ß ? more weight on the network overall
  workload
• larger ? ? less flooding overhead among
  superpeers
• The optimal size ratio increase as a decrease

38
Performance Evaluation

• The effectiveness of DLM 1
• (Left) from 300th, with the lifetime with halved
  mean value
• (Right) from 1000th, the capacity with doubled
  mean value
• Almost constant ratio is maintained.

39
Performance Evaluation

• Effectiveness of different approaches and
  policies
• DLM-2 performs the best since it employs the
  information exchange mechanism between
  neighboring peers
• DLM-3 performs the worst as it only runs on the
  superpeers.
• The performance on the other two metrics, average
  capacity and average age , also show similar
  features.

40
Performance Evaluation

• Periodic Exchange (PE) policy in DLM-2
• Different T have large effects on the performance
• PE (T30 min) lt PU (Pushing Exchange) lt PE
  (T5min)

41
Performance Evaluation

• Compare DLM-1 with preconfigured algorithm under
  dynamic network situations where the new peer
  mean capacity values are periodically changed

42
Discussion on the Side Effects of DLM

• Overhead for information exchanging
• The additional overhead is negligible
• Quite light-weight since transferred only between
  directly connected neighbors
• The frequency of DLM message transferring is
  quite low
• Piggybacked
• Peer Adjustment Overhead (PAO)
• When a superpeer is demoted to be a leaf-peer, it
  cut the connections to the leaf-peers, and the
  leaf-peers disconnected will try to connect to
  another superpeer instead.
• The ration of POA to NLCO is trivial in the
  real-world P2P networks with millions of peers
• (NLCO New Leaf-initiated Connection Overhead)

43
Conclusion

• Contribution
• Propose a workload model by analyzing the
  workloads on one superpeer and on the total
  network
• Obtain an optimal layer size ration that can
  minimize the weighted workload of the network
• Propose a dynamic layer management algorithm,
  DLM, which can adaptively elect peers and adjust
  them between superlayer and leaf-layer.
• Simulation result
• Simulation results show that DLM can maintain a
  given size ratio of superlayer to leaf-layer and
  designate peers with long lifetime and large
  capacities as superpeers and the peers with short
  lifetime and low capacity as leaf-peers under
  highly dynamic network situations.
• The meaning of DLM
• DLM is completely distributed in the sense that
  each peer run DLM independently.
• With the support of DLM, the quality of a
  superpeer system can be significantly improved.

44
Related Work

• Modeling unstructured P2P networks
• Ge et al.7
• developed a simple mathematical model to
  illustrate the performance issues of P2P systems
• apply the model to three different peer-peer
  architectures centralized indexing, distributed
  indexing with flooded queries, and distributed
  indexing with hash-directed queries
• Menasce and Kanchanapalli 14
• proposed a P2P protocol
• developed an analytical model to analyze the
  performance
• The studied metrics include the success rates,
  the fraction of peers involved in a search, and
  the average number of hops required to find a
  directory entry
• Cooper and Garcia-Molina 5
• Proposed a search/index links (SIL) model for P2P
  search network
• The parallel search clusters are superior to
  existing superpeer networks under some metrics

45
Related Work

• Structured P2P protocols
• presented, along with mathematical analysis
  18232717
• Tree-based model was proposed, the routing
  adaptively feature was analyzed 11
• Xu et al.24 studied the fundamental tradeoff
  between the routing table size and the network
  diameter by modeling.
• Applications based on superpeer architecture
  KaZaA
• 37 of all TCP traffic, more than twice the Web
  traffic on the University of Washington, in June
  2002
• 76 of P2P traffic belong to KaZaA/FastTrack
  traffic, only 8 comes from Gnutella in US.
• KaZaA is proprietary and uses encryption
• little has been known about the protocol
• no layer management mechanism is proposed to date.

46
Related Work

• Superpeer system
• Yang and Garcia-Molina 26
• Examined the performance trade-off by considering
  superpeer redundancy and topology variations
• Studied drawbacks of superpeer networks and
  reliability issues
• Singla and Rohrs 22
• To make the Gnutella networks more scalable,
  described how ultrapeers work in an ideal network
  with a static topology and a handshaking
  mechanism based on the Gnutella v0.6 protocol
• in which some requirements for superpeers were
  proposed, such as not firewalled, suitable os,
  sufficient bandwidth, and sufficient uptime.
• Singh et al. 21
• Presented some incentives to deploy superpeers
  and proposed a topic-based search scheme to
  increase the effectiveness of superpeer