The GRIMOIRES Service Registry

1
The GRIMOIRES Service Registry

• Weijian Fang
• School of Electronics and Computer Science
• University of Southampton

2
The Grimoires Registry

• Grid RegIstry with Metadata Oriented Interface
  Robustness, Efficiency, Security
• A Grimoire is a magician's manual for invoking
  demons (Oxford English Dictionary)
• Pronunciation grimwar / grimwars
• The Grimoires registry hosts descriptions of
  services and workflows, which a scientist can use
  for forming complex experiments, for discovering
  and for invoking them.
• Initially developed in the context of the myGrid
  Project (www.mygrid.org.uk)
• Currently, a managed program of Open Middleware
  Infrastructure Institute (www.omii.ac.uk)

3
Team Members

• Prof. Luc Moreau
• Victor Tan
• Simon Miles
• Weijian Fang
• Previous members
• Sylvia Wong
• Vijay Dialani
• Juri Papay

4
Outline

• Service Registry and UDDI
• Grimoires Approach
• Performance Evaluation

5
Outline

• Service Registry and UDDI
• Grimoires Approach
• Performance Evaluation

6
Service Discovery

• Service-Oriented Architecture
• There are a large number of deployed services.
• How to efficiently and precisely discover a
  service?
• Service registry
• Service publication
• Service discovery

7
UDDI

• Universal Description, Discovery and Integration
• The service publication and discovery spec
  adopted by WS community.
• The latest version is v3. But v2 is still widely
  used.
• The UDDI standard defines
• A data model to describe services
• A set of APIs to publish/discover services

8
UDDI Data Model
9
An Example Google Web Search

• ltbusinessEntity
• businessKey"A4B950A0-5A0A-11D7-8130-B77626E42CCD
  "gt
•   ltnamegtGoogle Busines Entitylt/namegt
•   ltdescriptiongt
• Google Search Company
• lt/descriptiongt
• ltbusinessServicesgt
• ltbusinessServicegt lt/businessServicegt
  lt/businessServicesgt
• ltcategoryBaggt lt/categoryBaggt
•   lt/businessEntitygt

10
An Example Google Web Search

• ltbusinessService businessKey"A4B950A0-5A0A-11D7-8
  130-B77626E42CCD" serviceKey"60E32D00-5A0B-11D7-8
  130-B77626E42CCD"gt
•   ltnamegtWeb Searchlt/namegt
•   ltdescriptiongt
• This service provides World-Wide Web Advanced
  Search
• lt/descriptiongt
• ltbindingTemplatesgt
• ltbindingTemplate bindingKey"AB717690-5A12-11D7
  -8130-B77626E42CCD" serviceKey"60E32D00-5A0B-11D
  7-8130-B77626E42CCD"gt
• ltaccessPoint URLType"http"gt
• http//api.google.com/search/beta2
• lt/accessPointgt
• lttModelInstanceDetailsgt
• lttModelInstanceInfo
• tModelKey"UUID5C857830-5A0B-11D7-8130-B77626E
  42CCDgt
• lt/tModelInstanceInfogtlttModelInstanceDetailsgtlt/bind
  ingTemplategtlt/bindingTemplatesgtlt/businessServicegt

11
An Example Google Web Search
tModel tModel
Key UUIDC1ACF26D-9672-4404-9D70-39B756E62AB4
Name uddi-orgtypes
Description UDDI Type Taxonomy
OverviewDoc URL http//www.uddi.org/taxonomies/UDDI_Taxonomy_tModels.htmUDDItypes
OverviewDoc Description Taxonomy used to categorize Service Descriptions. 
Value wsdlSpec Specification for a Web Service described in WSDL

• lttModel tModelKey"UUID5C857830-5A0B-11D7-8130-B7
  7626E42CCDgt
• ltnamegturnGoogleSearchlt/namegt
• ltoverviewDocgt
• ltoverviewURLgt
• http//api.google.com/GoogleSearch.wsdl
• lt/overviewURLgt
• lt/overviewDocgt
• ltcategoryBaggt
• ltkeyedReference tModelKey"UUIDC1ACF26D-9672-44
  04- 9D70-39B756E62AB4" keyName"wsdlSpec"
  keyValue"wsdlSpec"/gt
• lt/categoryBaggt
• lt/tModelgt

12
An Example Google Web Search

• ltbusinessEntity
• businessKey"A4B950A0-5A0A-11D7-8130-B77626E42CCD
  "gt
•   ltnamegtGoogle Busines Entitylt/namegt
•   ltdescriptiongt
• Google Search Company
• lt/descriptiongt
• ltbusinessServicesgt
• ltbusinessServicegt lt/businessServicegt
  lt/businessServicesgt
• ltcategoryBaggt lt/categoryBaggt
•   lt/businessEntitygt

13
An Example Google Web Search
tModel tModel
Key UUIDC0B9FE13-179F-413D-8A5B-5004DB8E5BB2
Name ntis-govnaics1997
Description Business Taxonomy NAICS (1997 Release)
OverviewDoc URL http//www.uddi.org/taxonomies/UDDI_Taxonomy_tModels.htmNAICS
OverviewDoc Description This tModel defines the NAICS industry taxonomy.
Value 514191 On-Line Information Services

• ltcategoryBaggt
• ltkeyedReference
• keyName"On-Line Information Services"
  keyValue"514191" tModelKey"UUIDC0B9FE13-179F-
  413D-8A5B-5004DB8E5BB2" /gt
•   ltkeyedReference
• keyName"Web search engine providers"
  keyValue"81.11.21.04.00" tModelKey"UUIDCD1532
  57-086A-4237-B336-6BDCBDCC6634" /gt
•   lt/categoryBaggt

14
What UDDI lacks

• UDDI is only a contact point to lots of
  descriptions elsewhere, expressed in different
  formalisms.
• Users have to retrieve, parse, understand, reason
  over those distributed descriptions.
• Rigid data model with limited extensibility
• tModel as metadata is difficult to use, and not
  expressive.
• Prevent customized service description.
• WS technical interface is expressed as an
  outgoing link (through tModel).
• Prevent discovery based on technical interface
  detail.

15
Simply Registering WSDL Not Work

• lt?xml version"1.0" encoding"UTF-8" ?gt
• ltwsdldefinitions targetNamespace"http//www.ebi
  .ac.uk/alignmentblastn_ncbiderived"
  xmlns"http//schemas.xmlsoap.org/wsdl/"
  xmlnswsdl"http//schemas.xmlsoap.org/wsdl/"gt
• ltwsdlmessage name"runRequest1"gt 
• ltwsdlpart name"in0" type"xsdstring" /gt  
• ltwsdlpart name"in1" type"xsdstring" /gt  
• lt/wsdlmessagegt 
• ltwsdlmessage name"runResponse1" /gt
• ltwsdlportType name"AnalysisWSAppLabImpl"gt
• ltwsdloperation name"run" parameterOrder"in0
  in1"gt 
• ltwsdlinput message"implrunRequest1"
  name"runRequest1" /gt  
• ltwsdloutput message"implrunResponse1"
  name"runResponse1" /gt  
• ltwsdlfault message"implSoaplabException"
  name"SoaplabException"/gt
• lt/wsdloperationgt 
• lt/wsdlportTypegt 
• lt/wsdldefinitionsgt

Inputs Details
No information about the meaning of these strings!
Nor about the meaning of the operation!
16
Other Related Work

• OWL-S
• Formerly DAML-S
• A service description data model
• Uses the OWL web ontology language to describe a
  Web Service from three component perspectives
• what it does (service profile),
• how it works (service model),
• and how to access it (service grounding).
• GT MDS
• The Monitoring and Discovery System in Globus
  Toolkit
• Focus on the mechanism to disseminate and gather
  information on Grids rather than the information
  model to describe services or resources.

17
Outline

• Service Registry and UDDI
• Grimoires Approach
• Performance Evaluation

18
RDF Representation

• Grimoires represents ALL information in RDF
  triples.
• Resource Description Framework
• (subject, predicate, object)
• Benefits using RDF
• There are no longer links to remote documents
• Things can be easily glued together
• It offers us a uniform querying/reasoning
  capability
• Ontology reasoning is a bonus

19
Metadata Annotation

• Metadata annotation helps service discovery.
• Customized service description
• Service consumers can share experience in service
  discovery and invocation
• Metadata examples
• (Service, UseResource, aDatabase)
• (InputMessage, SemanticType, NucleotideSequence)

20
Grimoires Functionality

• UDDI interface
• UDDIv2 compliant
• WSDL interface
• Publish and inquire for the WSDL descriptions,
  for instance,
• Publish a WSDL by its URL
• Find the input message of an operation.
• Given an operation, find out which WSDL
  description it belongs to.

21
Grimoires Functionality (contd.)

• Metadata interface
• Attach a piece of metadata to UDDI entities
  (business and service), WSDL entities (operation
  and message part), metadata.
• The metadata can be a String, a URI (representing
  an ontology concept), or RDF triples (complicated
  ontology description).
• Published by either service provider or a third
  party.
• Retrieve metadata attached to an entity.
• Discover entity by metadata.
• All information is uniformly represented in RDF.

22
(No Transcript)
23
Grimoires Architecture
UDDIv2 WS API
Metadata WS API
WSDL WS API
Lifetime Management
Access Control
Notification
RDF Serializer
RDF Deserializer
RDF Querier
Jena RDF API
Jena-Sesame-Model
Jena DB Store
Jena File-backed In-memory Store
Sesame DB Repo.
Sesame File-backed In-memory Repo.
Sesame Native Repo.
24
File-backed In-memory RDF Store

• The default RDF store in Grimoires
• Efficient in query in-memory
• Persistent file-backed
• Adjustable sync delay
• Larger sync delay possible to batch multiple
  writes to the file
• Smaller sync delay more resilient to fault
• Scalable enough 1GB memory for 25000 normally
  heavy service descriptions

25
Lifetime Management

• Each published entity can have a lifetime.
• Some action can be taken when the lifetime is
  over.
• Automatically clean up the registry, and avoid
  the accumulation of obsolete services.
• TerminationTime metadata
• There is a default termination time for all
  entities
• Can also be set or renewed by users at each
  entity
• TerminationAction metadata
• Deletion
• Deprecation
• Hidden, can be recovered later.
• There is a default termination action for all
  entities
• Can be overridden at each entity

26
Authentication and Access Control

• Authentication is based on signature creation and
  verification defined in WS-Security
• Uses X509 certificate
• Client-side SOAP message signed server side
  verifies and extracts X509DN
• A simple access control method
• Who can conduct what action
• Based on a list of (X509DN, operations allowed)

27
Authentication and Access Control

• A XACML-based access control method
• Who can conduct what action on which resource
• X509DN and UDDI request formatted into XACML
  request (subject, resource, action)
• Request is evaluated against existing XACML
  policy in XACML engine of Grimoires
• Can be mapped into role-based context
• Ongoing work on allowing XACML statements to be
  provided by registry clients as metadata
  attachments.

28
Grimoires Release

• Open source implementation
• Tested by OMII
• Released in OMII 2.1
• What are included in Grimoires release
• Grimoires registry
• Grimoires client side Java API
• GShell command line client
• junit test suite
• Well organized and abundant documentation
• Interoperable with standard UDDI tools
• UDDI browser, uddi4j
• Deployment environments
• OMII 2.x, GT4, and standard Tomcat/Axis

29
Outline

• Service Registry and UDDI
• Grimoires Approach
• Performance Evaluation

30
Benchmarking Methodology

• Two fundamental operations in a service registry
  service publication and discovery.
• We want to investigate to what extent the
  publication and discovery overheads are affected
  by the registry data size.
• While (true)
• publish 100 different services
• measure publication overhead
• inquire for 100 random services against all
  published services
• measure inquiry overhead

31
Benchmarking Methodology

• Publish a service
• UDDI save_business
• UDDI save_tModel
• UDDI save_service
• Metadata addMetadataToEntity (semantic discovery
  only)
• Inquire for a service
• UDDI find_service or Metadata find_service (by
  metadata, for semantic discovery)
• UDDI get_serviceDetail

32
Experiments

• Grimoires vs. jUDDI 0.9.4
• Memory usage of (file-backed) in-memory Grimoires
• Grimoires Performance when using persistent
  stores
• Grimoires Performance of WS-Security
• metadata publication and discovery by metadata in
  Grimoires

33
Grimoires vs. jUDDI Publication
jUDDI 226.3 ms to publish one service when there
are 2000 in the registry.
Grimoires 37.8 ms to publish one service when
there are 2000 in the registry.
34
Grimoires vs. jUDDI Inquiry
Grimoires 56.9 ms to discover one service when
there are 2000 in the registry.
But Grimoires provides what UDDI does not
provide uniform query, metadata annotation,
and service discovery by metadata.
jUDDI 38 ms to discover one service when there
are 2000 in the registry.
35
Memory Usage
About 1GB memory for 25000 services
36
Persistent Stores Publication
PostgreSQL 37066 ms to publish one service when
there are 2000 in the registry.
File-backed in-memory 62.4 ms to publish one
service when there are 2000 in the registry.
37
Persistent Stores Inquiry
PostgreSQL 2173 ms to discover one service when
there are 2000 in the registry.
File-backed in-memory 42.7 ms to discover one
service when there are 2000 in the registry.
38
WS-Security Publication
Signed 1532 ms to publish one service when there
are 2000 in the registry.
Not signed 88 ms to publish one service when
there are 2000 in the registry.
39
WS-Security Inquiry
Signed 987 ms to discover one service when there
are 2000 in the registry.
Not signed 98.9 ms to discover one service when
there are 2000 in the registry.
40
Metadata Publication
Attach metadata
Publish service
41
Discovery by metadata
Inquiry-by-name 56.9 ms to discover one service
when there are 2000 in the registry.
Inquiry-by-metadata 80.5 ms to discover one
service when there are 2000 in the registry.
SELECT ?key WHERE (?service, ltrdftypegt,
ltuddiBusinessServicegt) (?service0,
ltuddihasNamegt, ?nameBag) (?nameBag, ?v, a good
service") (?service0, ltuddihasServiceKeygt, ?key)
SELECT ?key WHERE (?service, ltrdftypegt,
ltuddiBusinessServicegt) (?service0,
ltmetahasMetadatagt, ?metadataBag) (?metadataBag,
?v, ?metadata) (?metadata, ltmetahasMetadataTypegt,
ltmetauseResourcegt) (?metadata,
ltmetahasMetadataValuegt, blastn
database) (?service0, ltuddihasServiceKeygt, ?key)
42
Summary

• Grimoires is a UDDIv2 compliant service registry.
• Grimoires provides important capabilities that
  are not provided by UDDI.
• Uniform publication/discovery
• WSDL registration/discovery
• Metadata annotation
• Discovery by metadata
• Grimoires is efficient.
• Future work
• Notification
• WS-Eventing, WS-Notification
• WSRF
• Integration with GT4

43
Download Grimoires

• Grimoires homepage
• www.grimoires.org
• Grimoires release
• http//twiki.grimoires.org/bin/view/Grimoires/Soft
  wareReleases
• The latest version is 1.0.4.

44
(No Transcript)