Searching PCCE Jabber Archives

1
C O M P U T A T I O
N A L R E S E A R
C H D I V I S I O N
Searching PCCE Jabber Archives Michael Tobias 20
Aug 2004
2
Overview

• Background
• Challenges
• Solutions
• Future Work

3
Background

• Jabber is a set of protocols for streaming of XML
  elements between any two points on a network
• We use it for IM (its original, and still main,
  application)
• Already have Python 'bot' that archives chats
• Just invite bot to chatroom

4
Challenges

• Improve methods for storing XML produced by
  archive bots
• Solution XML Database
• Improve search utility
• Solution XQuery
• Present search results in a clear and useable
  manner
• Solution XSL Stylesheets

5
Solutions

• Improve methods for storing XML produced by
  archive bots
• Solution XML Database
• Improve search utility
• Solution XQuery
• Present search results in a clear and useable
  manner
• Solution XSL

6
Solutions

• Improve methods for storing XML produced by
  archive bots
• Solution XML Database
• Improve search utility
• Solution XQuery
• Present search results in a clear and useable
  manner
• Solution XSL

7
Native XML databases

• Designed from the ground up to store XML data
• Don't have to map your XML to some other data
  structure
• Just insert the data as XML and retrieve it as
  XML
• Gain flexibility through the semi-structured
  nature of XML and the schema independent model

8
Native XML databases eXist

• eXist An Open Source Native XML database
• Lightweight
• Written completely in Java
• Easily deployed in a number of ways
• stand-alone server process
• inside a servlet-engine
• directly embedded into an application
• Tight integration with existing XML development
  tools

9
Native XML databases eXist

• eXist (continued)
• Java Binding
• Extension modules
• HTTP support
• Database manipulation
• XSL transformations
• Efficient index-based XQuery processing
• Full-text search functions

10
Solutions

• Improve methods for storing XML produced by
  archive bots
• Solution XML Database
• Improve search utility
• Solution XQuery
• Present search results in a clear and useable
  manner
• Solution XSL

11
XQuery Overview

• An XML query language
• Provides features for retrieving and interpreting
  information from many types of XML data sources
• XQuery Version 1.0 is an extension of XPath
  Version 2.0

12
XQuery Syntax

• Example

13
XQuery and eXist

• XQuery support in eXist makes it possible to
  write entire web applications with just XQuery
  and XSLT.
• XQuery files can be directly passed to the
  database, using either
• the XQueryServlet,
• the XQueryGenerator for Cocoon
• or the REST-style API

14
Solutions

• Improve methods for storing XML produced by
  archive bots
• Solution XML Database
• Improve search utility
• Solution XQuery
• Present search results in a clear and useable
  manner
• Solution XSL

15
XSL

• XSL - XML Stylesheet
• a vocabulary for specifying formatting

16
Results

• Demo
• Basic Search
• Advanced Search

17
Basic Search
18
Advanced Search
19
Results Overview
20
Results Details
21
Where to go from here

• Finish up Advanced Search
• Integrate w/other Jabber utilities
• Security

22
Work on other Jabber utilities

• Transfer of jabber registration to new server
• test.dsd.lbl.gov (crd.lbl.gov)
• Clean-up of some problems w/these pages
• remove user now working
• user warned of removal delay
• required fields presented first
• required fields indicated as such
• no more stack traces
• helpful errors when required fields are missing
• helpful errors on removal of invalid
  name/password

23
References

• http//exist.sourceforge.net/index.html
• http//www.w3.org/TR/xquery/