Versus: A Web Repository Daniel Gomes, Jo

1
Versus A Web RepositoryDaniel Gomes, João P.
Campos, Mário J. SilvaXLDB Research
GroupUniversity of Lisbondcg, jcampos,
mjs_at_di.fc.ul.pt

• Versus is a repository that enables storage,
  management and access to Web data.
• Versus enables quick development of Web
  applications that need to process large amounts
  of information in a short period of time, using a
  simple and extensible JAVA API

2
1st Prototype Features

• Extensible meta-data
• Enables extension of meta-data information stored
  in Versus
• Extensible code
• to fulfill specific requirements Web processing
  applications, with minimum code generation.
• Well defined API
• provides access methods according to the needs of
  Web applications, supporting different levels of
  access and performance.

• Generic data model
• enables storage and management of multi-purpose
  Web data
• Time support
• enables past views of objects through a
  versioning system
• Scalability and distributed operation
• enables applications to increase processing
  capabilities, through the parallel processing of
  data partitions

3
Versus is especially useful for applications
that can profit from parallel processing of Web
data.
4
Building a Web Application with Versus

• 1. Define the applications unique identifier
• 2. Define the partitioning function
• 3. Define a conflict resolution policy
• 4. Define timeouts for the processing of working
  units
• 5. Write application-specific code for data
  processing using Versus API.

5
Data Model

• An object has a reference to a Web document
• A version is a snapshot of an object at a given
  instant
• A layer represents a time unit in the repository
• An objectKey is a property associated to an
  object and therefore to every version of it.
• A versionProperty is a property associated to a
  certain version of an object
• Xmeta-Data is a container for XML data associated
  with each version.

6
Operational Model

• Archive Workspace stores data permanently,
  keeping version history of objects
• Group Workspace maintains a shared view of the
  data common to all application threads
• Private Workspace provides local storage and
  fast access to one application thread.
• Working Units container for a partition of data,
  which can be independently processed
• Check In/Check Out operations that move the
  working units from one workspace to another.

7
Current Future Work

• New Versus
• Native XML meta-data
• Improve performance
• P2P storage server for massive data management
  and scalable performance
• Versus applications
• TUMBA! (our Web search engine for the Portuguese
  Web at http//www.tumba.pt)
• Web pages repository with history
• Tarântula V.2 (Web crawler)
• Tumbas Index Generator and PageRanker
• Performance measurement and analysis
• Validation of Versus API.
• XQuery XSLT engine tightly coupled with Versus
• Query engine with access methods and adaptive
  algorithms

8

• Research on data management. We study and develop
  systems for data analysis, information
  integration and user access to large quantities
  of complex data from heterogeneous sources.
• Current main project is TUMBA!, a new search
  engine for the Portuguese Web. Research on
• Integration with Semantic Web
• Location-awareness
• Web Archiving

http//xldb.fc.ul.pt