Distributed Monitoring of Peer-to-Peer Systems

1
Distributed Monitoring of Peer-to-Peer Systems

• By
• Serge Abiteboul, Bogdan Marinoiu
• Docflow meeting, Bordeaux

2
Outline

• The Monitoring Problem Approach
• A language for specifying monitoring tasks P2PML
• P2PMonitor System
• ActiveXML Stream Algebra
• Architecture of P2PMonitor
• Monitoring Plan Generation Query Rewriting
• Focus on Filtering
• Reusing running tasks
• Work in progress

3
The Monitoring Problem

• P2P systems are
• a popular support for content sharing
  communities, distributed applications
• highly dynamic (intense communications, content
  changing rapidly, peers come/leave)
• and
• difficult to observe
• Observation is important !
• error management diagnosis
• statistics gathering optimization issues
  the  busiest  peer in a network
• business applications billing quality of
  service
• Web surveillance

4
Is it possible to observe analyse a P2P system
?

• Difficult (if not impossible)
• in a centralized way

• Yes, in a distributed manner

5
Approach

• XML stream

• Detect events at the (monitored) peer level
• Data changes, Web service calls -gt alerters
• Each event is represented as an XML document
• XML Stream
• (distributed) XML stream processing system
• XML Streams are published

6
Outline

• The Monitoring Problem Approach
• A language for specifying monitoring tasks P2PML
• P2PMonitor System
• ActiveXML Stream Algebra
• Architecture of P2PMonitor
• Monitoring Plan Generation Query Rewriting
• Focus on Filtering
• Reusing running tasks
• Work in progress

7
P2PML statement structure

• XQuery FLWR flavour
• For maps streams to XML variables
• Let assigns new XML variables
• Where imposes conditions on events (filtering
  and join criteria)
• Return generates reports / restructures XML
• By specifies publication means
• in channels for inside system publication
• e-mails, Web pages, RSS feeds for outside
  system publication

8
P2PML statement example

• for c on local outCOM
• let timeCall c.call.time
• and duration c.response.time -
  timeCall
• where
• c.call.method GetTemp and duration gt
  10
• and c.call.site "http//meteofrance.fr"
• return ltlongGetTempgt
• ltcallTimegttimeCalllt/callTimegt
• ltdurationgtdurationlt/durationgt
• lt/longGetTempgt
• by channel QoSAlerts

9
Outline

• The Monitoring Problem Approach
• A language for specifying monitoring tasks P2PML
• P2PMonitor System
• ActiveXML Stream Algebra
• Monitoring Plans
• Architecture of P2PMonitor
• Focus on Filtering
• Reusing running tasks
• Work in progress

10
ActiveXML Stream Algebra

• is the support for monitoring plan representation
  and the basis for the its optimization
• Distribute the work among the peers
• Try to place computation close to data if
  possible
• Try to reduce redundancy

11
Scenario
P2PMLQueries
XML streams
12
Monitoring Plans
13
Architecture of P2PMonitor(1)

• Subscription Manager
• Alerters (WS Alerter, Database Alerter, RSS
  Alerter)
• Stream Processors
• Without  storage  Filter, Restructure, Union
• With  storage  Join, Group-By, Duplicate
  Removal
• Publishers
• E-mail, WebPage, RSS
• Channel Publisher a user or another peer may
  subscribe to it

14
Architecture of P2P Monitor(2)
15
Outline

• The Monitoring Problem Approach
• A language for specifying monitoring tasks P2PML
• P2PMonitor System
• ActiveXML Stream Algebra
• Monitoring Plans
• Architecture of P2PMonitor
• Focus on Filtering
• Reusing running tasks
• Work in progress

16
Focus on Filtering

• Filtering is a crucial operator in stream
  processing !
• E.g., Many users might be interested in events
  coming from the same source / alerter
  bottleneck hazard
• Our approach to the problem two-stage filtering
• Reasons
• Attributes of XML documents root reflect the
  most important properties of an event
• The events details can be given intentionnally
  (ActiveXML style)

17
Two-stage filtering

• A subscription is viewed as a conjunction of
  simple conditions (e.g.,  attribute 
   value ) and  more difficult  XPath queries
• 1st data structure regroups the (ordered) simple
  conditions of all the subscriptions by
  commonalities (Atomic Event Set structure)
• 2nd data structure regroups XPath queries of
  all the subscriptions (path based indexing
  YFilter style using NFA)
• On a XML document
• 1st stage read the root, evaluate AES, detect
  the  difficult  XPath queries that remain to be
  evaluated
• 2nd stage (if needed) adapt the second
  structure and evaluate the set of XPath queries
  on the body of the XML document (if necessary
  execute Web service calls).
• The output is the set of the subscriptions
   hit  by the XML document

18
Outline

• The Monitoring Problem Approach
• A language for specifying monitoring tasks P2PML
• P2PMonitor System
• ActiveXML Stream Algebra
• Monitoring Plans
• Architecture of P2PMonitor
• Focus on Filtering
• Reusing streams / running tasks
• Work in progress

19
Reusing running tasks

• Optimization by trying to avoid redundancy
• Before building new operators (and streams), try
  to discover useful ones
• Stream representation in XML
• Stream Definition Database description of
  available streams
• Distributed, not centralized (avoid bottlenecks)
• Implemented using KadoP index and repository
  system over a DHT

20
Stream replication and equivalence(1)

• Streams can be replicated between peers
• With two similar operators on two replicas of the
  same stream, we obtain two equivalent streams
• Replication can be represented in the Stream
  Definition Database
• Stream Equivalence is difficult to detect

21
Algorithm for discovering useful streams

• It uses XPath queries on the Stream
  Definition Database
• E.g. for identifying the output stream of
  alerter inCOM
• /Stream_at_PeerIdp1Operator/inCOM
  -gt(S1, P1)
• It goes bottom-up
• on the query tree
• E.g.,
• JoinP(sF(inCOM_at_P1),
• outCOM_at_P2)
• (S5, P1)

22
Outline

• The Monitoring Problem Approach
• A language for specifying monitoring tasks P2PML
• P2PMonitor System
• ActiveXML Stream Algebra
• Monitoring Plans
• Architecture of P2PMonitor
• Focus on Filtering
• Reusing streams / running tasks
• Work in progress

23
Work in progress (1)

• Link with Incremental View Maintenance
• Defining a monitoring task by a tree-pattern
  query on an active document with streams -
  powerful way of expressing complex monitoring
  tasks (difficult to express directly in P2PML)

24
Work in progress (2)

• Introducing explictly the  time  in P2PML
  -possible impact on P2PMonitor performance
  (reactivity) and resource consumption (needed
  storage)
• E.g.
• for e1 on P1inCOM, e2 on P2outCOM
• where e1.timeEvent gt e2.timeEvent 25
• Queries on traces obtained by P2PMonitor
  diagnosis, detecting patterns of evolution for
  the monitored system
• E.g. Trace I1,I2In - instances of a
  document
• For each new order detected in instance Ik, there
  is a payment present in one of the following
  instances

25

• Thank you very much!