Semantic Logging using the Resource Description Framework

1
Semantic Logging using the Resource Description
Framework
Mark A. Baker and Richard J. Boakes Distributed
Systems GroupUniversity of PortsmouthCCGrid
Special Session for Work-in-Progress Papers on
Novel Grid Technologies9 - 12 May 2005
(Cardiff, UK)
2
Overview

• The challenges in evaluating distributed systems
  are similar to those of sequential and parallel
  systems
• A flexible infrastructure to unify recorded
  information about distributed systems would
  enable a global-view be analysed
• Naïvely merging records is not sufficient
  syntactic and semantic translation is necessary
• We are investigating the use of Semantic Web
  technologies to achieve this integration

3
Distributed Systems

• We wish to unify and evaluate distributed systems
  that are heterogeneous in nature
• Multiple
• sites,
• hosts,
• programs
• software layers
• logging systems
• Types/format of record
• We must also consider
• Multiple ownership
• Dynamic discovery of participants

4
Current Evaluation Tools

• Differing purpose
• Debugging, performance tuning
• Differing Focus
• Sequential and parallel applications

5
Types of Record

• System records
• CPU load, memory management, network usage and
  errors
• Middleware records
• Daemon/server logs
• Application records
• Specialised instrumentation logs(e.g. trace and
  profile information)
• Plus potentially
• External monitor logs (e.g. DNS, traceroute)

6
Existing Record Formats

• Many formats to consider, e.g.
• LBLs NetLogger
• PSUs PPerfDB
• Common Logging Format (CLF)
• Others (tell us what and why)
• We cannot consider all record formats at design
  time so we must expect and allow for
  retrospective integration of formats

7
Semantic Logging

• Using Semantic Web standards we can
• Combine different record formats
• Integrate new formats retrospectively
• Incorporate metadata
• At any granularity and stage.

8
RDF (Resource Description Framework)

• Describes each resource using a series of
  explicit statements.
• An RDF model consists of many statements about
  many resources.
• Each statement contains three parts
• A subject identifies the resource that is being
  described
• A predicate describes the type of information
  being communicated
• An object describes the value being attributed

9
CLF Example
10
Work Completed

• Defined RDF schemas for
• A Generic Unified Log Format (GULF),
• The Common Log Format (GULF-CLF).
• Extended the RDFX Framework
• Written a CLF to GULF-CLF translation plug-in,
• Extended UI to recognize GULF-CLF elements.

11
Work in Progress

• Further Schema Development
• Granularity, data formats, annotations
• Semantic Knowledge Organisation Systems (SKOS)
• Reusable vocabularies
• Translation Plug-in for NetLogger
• Established and understood
• Sample NetLogger data promotes rapid development.
• Graphical representation of the data model
• Investigating existing profiling tools
• Multiple visualisation within the framework
• Scalable Vector Graphics (SVG) and OpenGL

12
CLF Records in RDFX
13
A Clusters Activity in SVG
14
OpenGL based UI
15
Distributed Systems Group

• Email
• rich_at_boakes.org
• mark.baker_at_computer.org
• Websites
• http//dsg.port.ac.uk/
• http//rdfx.org

16
Extended Discussion Slides

• The following slides are not part of the main
  presentation but exist as examples for use in
  broader discussion.

17
CLF Example (Extended)
18
CLF Example (Extended)

• Assume we have no access to or knowledge of to
  example.org systems.
• A web application is provided by example.org.
• Our distributed system discovers the web
  application dynamically.
• Round Robin DNS load balancing is used by the
  provider.
• Clients Connect to http//example.org/webapp/test
• One of the machines is running unpatched code
  which does not error, but returns slightly
  different results.
• Errors are erratic

19
CLF Example (Extended)
20
NetLogger File Format

• The following slides describe an Event Record in
  the NetLogger File Format.

21
NetLogger File Format
22
NetLogger File Format
23
NetLogger File Format
24
NetLogger File Format
25
NetLogger File Format
26
RDFX Background

• Requirements
• Open extensible platform for tool integration
• Open source
• OS Independent
• Multiple language support
• Rapid development
• Eclipse
• www.eclipse.org

27
A Framework for Unification
28
Etnuss TotalView