The German Astrophysical Virtual Observatory (GAVO)

1
The German Astrophysical Virtual Observatory
(GAVO)

• Knowledge Networking for Astronomy in Germany and
  abroad
• Gerard Lemson1,2, Wolfgang Voges1, Joachim
  Wambsganss2
• GAVO team

1 Max-Planck-Institut für extraterrestrische
Physik, Garching 2 Astronomisches Rechen
Institut/Zentrum für Astronomy der Universität
Heidelberg
2
Overview

• Knowledge networking for astronomy the Virtual
  Observatory
• Standardisation the International Virtual
  Observatory Alliance (IVOA)
• The Virtual Observatory in Germany GAVO
• Theory in the VObs

3
The status quo

• Astronomy produce a huge range of valuable data
  products,
• Of sometimes astronomical sizes (soon
  Petabytes/yr).
• From ground-based observatories and satellites
• From all-sky surveys, and from large variety of
  individual targeted observations.
• In all wavelengths.
• Dont forget computer simulations
• Requires large variety of disciplines/specialisati
  ons to produce and analyse.
• Data products are relatively quickly made public
  (1 yr).
• Many already available as online archives, more
  or less standardised, homogeneous.
• In combination promises interesting new science.

4
Some astronomical data products
Images
Spectra
Catalogues
Simulations

• John Hibbard http//www.cv.nrao.edu/jhibbard/n40
  38/n4038.html
• NASA/CXC/SAO/G. Fabbiano et al.
• Di Matteo, Springel and Hernquist, 2005

5
(No Transcript)
6
But

• Where is the data?
• What do these archives contain?
• How can they be accessed?
• How do we analyse these (very) large datasets?
• How can we combine them?
• ...
• The Virtual Observatory (VObs) is the proposed
  answer to these questions.

7
The VObs ...

• Requires
• archiving
• curation
• online availability
• description
• access tools
• filtering tools
• analysis tools
• standardisation

• Promises
• improved communication
• reuse of results
• validation
• comparison
• combination
• federation
• collaboration
• new science

? O(16) national VObs projects, organized in the
IVOA
8
IVOA
9
Mission statement

• Facilitate the international coordination and
  collaboration necessary for the development and
  deployment of the tools, systems and
  organizational structures necessary to enable the
  international utilization of astronomical
  archives as an integrated and interoperating
  virtual observatory.

10
IVOA organisation

• Since 2002
• Activities divided in working - and interest
  groups, tiger teams
• Meet twice a year in interoperability meetings
  (two weeks from now, Beijing)
• Active mailing lists, wiki pages
• Standardised standardisation process

11
IVOA activities

• Working groups create standards for
• publication and discovery (Resource Registry)
• (meta-)data description (DM, Semantics)
• selection and remote filtering (DAL, VOQL)
• formats for transmitted data (VOTable)
• (web) services, distributed workflows (GWS)
• application interoperability (Applications)
• event notification (VOEvent)
• interest groups represent special interests
• Data curation
• Grid
• Theory

12
Some results

• Resource registry data models various
  implementations
• VOTable format (XML schema) many support tools
• Data models for
• space-time coordinate systems
• characterisation of observations
• spectra
• Simple data access protocols for
• source catalogues
• 2D images
• 1D spectra
• In development
• Astronomical data query language (ADQL)
• Astronomical web service standards (UWS)
• Application messaging
• Simulation data models access protocols

13
The German Astrophysical Virtual Observatory

• http//www.g-vo.org

14
GAVO I 2002-2005

• BMBF funded
• Partners
• Astrophysikalisches Institut Potsdam (AIP)
• Astronomisches Rechen Institut/Zentrum für
  Astronomy der Universität Heidelberg (ARI)
• Max-Planck-Institut für extraterrestrische Physik
  (MPE), Garching
• Hamburger Sternwarte
• Asociated partner Max-Planck-Institut fur
  Astrophysik (MPA), Garching
• Activities
• RD
• Prototyping
• Special attention
• Archive publication ROSAT, RAVE
• Data mining cross-matching, classification
• Grid computing simulations, distributed cluster
  finder
• Theory virtual telescopes, archiving
  simulations, IVOA

15
GAVO II 2006-2008

• BMBF funded
• Partners
• AIP, MPE, MPA,
• Technische Universität Munchen-Informatik
• Universität Tübingen
• Focus move to scientifically useful services
• Projects
• Millennium database (see later)
• IVOA representation (theory, VOQL)
• Standard services (SIA, SSA,SCS)
• Custom services
• VObs expertise center at ARI ...

16
VObs expertise center _at_ ARI

• IVOA compatible metadata repository for community
• Implementation data access query protocols
• Storage (smaller) data sets, especially science
  ready data
• Tools
• Outreach/PR
• Help-desk

17
Theory in the VObs some observations

• Simulations not as simple as observations
• less homogeneous
• complex observables
• no standardisation on data formats
• archiving ad hoc, for local use
• Current IVOA standards somewhat irrelevant
• no common sky
• no common objects
• requires data models for content, physics, code
• Moores law for N-body) simulations
• Very large simulations possible
• NB also makes useful lifetime relatively short

18
Moores law for N-body simulations
Courtesy Simon White
19
Virgo collaborationsMillennium database

• Largest cosmological simulation to date
• 10 billion particles evolving under gravity
• 500 Mpc (2Gly) box
• 64 snapshots
• 350000 CPU hours
• O(30Tb) raw data
• Derived data
• density fields
• clusters, merger trees
• galaxies, merger trees
• realistic, observed galaxy catalogues

Courtesy Volker Springel
20
Time evolution merger trees
21
Real and Mock catalogues
Courtesy Volker Springel
22
Database web server

• Derived data products only
• SQLServer database
• Apache web server
• portal http//www.mpa-garching.mpg.de/millennium/
  
• public DB access http//www.g-vo.org/Millennium
• private access http//www.g-vo.org/MyMillennium
• Access methods
• browser producing various formats, plotting
  capabilities
• stream based wget IDL, R, etc allows
• finite query time (30sec-7min)
• Features
• efficient tree storageaccess
• spatial indexing
• MyDB

23
(No Transcript)
24
(No Transcript)
25
Usage statistics

• Up since Aug 2006
• Community notified via preprint server
  http//xxx.lanl.gov/abs/astro-ph/0608019
• 130 registered scientific users
• gt1.4 million individual SQL queries
• gt 4 billion returned rows (since March 8 2007)

26
Summary

• VObs is natural extension of astronomys history
  of data archiving, standardisation, online and
  open access.
• IVOA active, questions are complex technically,
  but politics are sometimes even harder.
• GAVO relatively small, but has found some niches,
  particularly theory.
• To be successful requires use by non-VO
  scientists, requires proper PR.

27
Thank you.
Further thanks to Volker Springel, Simon White,
Gabriella DeLucia, Jeremy Blaizot, Manfred
Kitzbichler (MPA, Garching), Carlos Frenk, John
Helly , Richard Bower (ICC, Durham, UK), Alex
Szalay (JHU, Baltimore) Opening picture courtesy
of NASA Goddard Space Flight Center.