Publishing and Resource Discovery with Registries

1
Publishing and Resource Discovery with Registries

• Ray Plante
• Gretchen Greene

2
All about Registries

• Overview of the Registry Framework
• Publishing to the NVO
• Visits to Registries (publishing)
• VOResource Resource Metadata in XML
• Visits to Registries (searching)
• IVOA Standard Registry Interface
• Exercise query registry in an application

3
The role of Resource Registries

• Used to discover and locate resourcesdata and
  servicesthat can be used in a VO application
• Resource anything that is describable and
  identifiable.
• Besides data and services organizations,
  projects, software,
• Presently concerned with simple set of resource
  types
• Registry a list of resource descriptions
• Expressed as structured metadata
• to enable automated processing and searching

4
An Overview of Data Discovery

• You can search the main NVO registry to find
  resources based on descriptive criteria
• NVO Registries are coarse-grained
• You can find organizations, archives, catalogs
• Wont find images, celestial objects, table
  records
• Registry framework contains multiple registries
• searchable registries
• publishing registries

5
Registry Framework
VO Projects
Full Searchable Registry
Data Centers
Local Searchable Registry
Specialized Portals Services
6
Registry Framework
VO Projects
harvest
(pull)
Full Searchable Registry
Data Centers
Local Searchable Registry
Specialized Portals Services
7
Registry Framework
VO Projects
harvest
(pull)
Cross-harvest
Full Searchable Registry
Data Centers
Local Searchable Registry
Specialized Portals Services
8
Registry Framework
VO Projects
harvest
(pull)
Cross-harvest
Full Searchable Registry
Data Centers
selective harvesting
Local Searchable Registry
Specialized Portals Services
9
Registry Framework
VO Projects
Full Searchable Registry
Data Centers
search queries
Local Searchable Registry
Client Applications
Specialized Portals Services
10
Registry Framework
VO Projects
Full Searchable Registry
Data Centers
search queries
Local Searchable Registry
Client Applications
Specialized Portals Services
11
NVO Public Registries
Private Publishing Registries

• HEASARC
• CDS
• Only support harvesting protocol

12
Overview of Publishing

• Resources are published if one can use NVO
  facilities to find them.
• How to Publish to the NVO
• http//us-vo.org/pub/files/PublishHowto.html
• Multiple layers of publishing
• Starts with registry description of resource
• Data Access Services
• Incremental exposure for incremental effort
• Who are you? How you publish depends on what you
  want to publish.
• An individual with a small data collection
• An archive center
• Someone with a cool service

13
Small collectionsVO-ready Repositories

• Repositories that allow users to deposit data to
  share with community
• Guarantee long-term storage, availability
• Automatic support for VO publishing mechamisms
• Entries into NVO Registry
• Support for standard services
• Cone Search, SIA, SSA, SkyNode
• Currently available Repositories
• Images NCSA Astronomy Digital Image Library
  http//adil.ncsa.uiuc.edu/
• Spectra Spectrum Service for the VO
  http//voservices.net/spectrum/
• More public repositories are expected to emerge
• Check NVO website (http//us-vo.org/) for latest

14
Persistent ArchivesTools for Federation

• Registering your resources with a VO publishing
  registry
• Enter description into registration form at one
  of the available NVO registries
• STScI/JHU Registry http//nvo.stsci.edu/voregist
  ry/
• NCSA Registration Portal http//nvo.ncsa.uiuc.ed
  u/nvoregistration.html
• Caltech Carnivore http//nvo.caltech.edu8080/ca
  rnivore/
• If you have a large number of resources to
  register, you can run your own registry on your
  own site
• Caltech Carnivore http//nvo.caltech.edu8080/ca
  rnivore/
• NCSA VORegistry-in-a-Box http//nvo.ncsa.uiuc.edu/
  VO/software/

15
Caution Construction Ahead

• Which registry should I register with?
• Behavior is slightly different
• Different strategies for making it easier to
  register multiple resources
• More uniform support is underway
• Support for standard services is strongest
• Cone Search, SIA, SkyNode
• SkyNode go to STScI or Carnivore
• SSA go to NCSA
• Your feedback is valuable!

16
Persistent ArchivesTools for Federation

• What can/should you register?
• Should your Organization
• Declares yourself as a publisher with an ID
• Should your Data Collection
• Users at least know how to access it via a
  Browser
• Can your existing services
• Browser-based services e.g. search page
• Traditional CGI services
• Web Services
• The next level
• Implement and register one or more standard
  services
• Cone Search
• Simple Image Access
• SkyNode
• Simple Spectral Access
• standard still in development

17
Cool ServicesIntegrating with the VO

• Register your service at a registry
• Currently as a generic resource
• Improved support for non-standard services coming
• (immediate future contact NVO project)
• Integrate support for standard VO formats,
  schemas
• FITS and VOTable
• Standard Data Model schemas (emerging)
• VOResource, Space-time Coordinates, Spectra
• Implement Standard Support Interface
• a standard in development for
• Self-description, tracking health and usage

18
A word about Identifiers

• IVOA Identifier a globally-unique URI
  identifying a resource
• Ex ivo//adil.ncsa/targeted/SIA
• Required as part of a registered resource
  description
• As publisher, you control what it looks like
• Two components
• Authority ID e.g. adil.ncsa
• Defines a namespace for identifiers
• Owned by a single publishing organization
• Resource Key e.g. targeted/SIA
• Name for the resource unique within the namespace
• Encourage re-use of local identifiers

19
Visits to the Publishing Registries

• Publishing Registries
• STScI/JHU Registry http//nvo.stsci.edu/voregist
  ry/
• NCSA Registration Portal http//nvo.ncsa.uiuc.ed
  u/nvoregistration.html
• Caltech Carnivore http//mercury.cacr.caltech.edu
  8080/carnivore/
• Recommend you stick with one registry
• Authority IDs are currently stuck to a single
  registry
• Cant do something? Contact us!

20
Resource Metadata XML Schema

• Classes of Resources
• Generic Resource
• Extensions e.g.
• DataCollection, Service, ConeSearch,
  SimpleImageAccess, SkyNode
• Organized into separate schemas
• Core resource metadata VOResource
• Various extensions schemas containing specific
  types
• Capable of describing
• Data centers, research organizations, missions,
  observatories
• Data collections, archives
• VO standard services Cone Search, Simple Image
  Access, Simple Spectral Access, SkyNode
• Existing Browser/CGI-based services
• Web Services

21
Searching the Registry

• Available Interfaces
• Keyword Search
• Intermediate Interfaces
• Form-based
• Under-development
• e.g. http//nvo.caltech.edu8080/carnivore/search
• Advanced SQL/ADQL/XQuery interfaces

22
IVOA Standard Registry Interface

• IVOA Working Draft 2 parts
• Harvesting
• sending descriptions from publishers to
  searchable registry
• Searching
• Searching
• methods
• keywordSearch(string words, boolean combineByOr)
• search(ADQLWhere constaints)
• Returns a list of VOResource descriptions
• Advanced searching with ADQL
• Just the where parti.e. search constraints--of
  ADQL
• In place of column names, use XPath to VOResource
  element
• Curation/Publisher like NASA
• Query extensible to any VOResource XML extension
• Maps readily to registry implementations based on
  RDBMS or XML-DB.

23
Why might a client use the standard search
interface?

• Uniform interface to all VO Registries
• Not dependant on a single registry
• Direct relationship between information you are
  querying and information you get back.
• Extensible to any type of resource description
• Re-use of ADQL
• Custom interfaces to Registries
• Registries can provide extended functionality
• More advanced capabilities
• Simpler interfaces for specialized purposes
• Client toolkits can provide simplifying
  interfaces
• Support for ADQL/s
• XPath aliases

24
What you might use this week

• STScI or Carnivore Registry portals to search for
  resources
• Any of the registration portals to register
  resources
• Software for querying registry from an
  application
• Java see java/dev/nvoregistry
• IDL see idl/VOlib_0.1 (call_registry.pro)
• Python see python/PythonVOClientLibrary.html
  (VORegistry.py)
• PHP see in php/web kwdSearch.php, phpVOlib