eSDO and the Grid

1
eSDO and the Grid
Using the Grid to Enhance UK Access to SDO Data
and Algorithms Elizabeth Auden Mullard Space
Science Laboratory 19 September 2005
2
The eSDO Project
SDO Solar Dynamic Observatory eSDO PPARC
grant PP/C502930/1, funded to provide virtual
observatory access to SDO data and algorithms for
the UK solar community. Phase A research 1
October 2004 30 September 2005 Phase B
development 1 October 2005 30 September
2007 Three Goals 1. Data centres in US and
UK 2. Eleven solar algorithms 3. Visualization
techniques to navigate huge dataset
3
Background
SDO Solar Dynamic Observatory SDO will launch
in August 2008 as part of NASAs Living With a
Star program. Raw data 2TB / day. AIA
Atmospheric Imaging Assembly this instrument
will provide 10 channels for multi-wavelength
observations of the Sun in optical and UV
frequencies. HMI Helioseismic and Magnetic
Imager the magnetic imager will look deep into
the Suns interior. Scientists will be able to
compare a number of magnetic field data products
with white light images.
WHITE SANDS New Mexico
STANFORD California
UK data centre
Fig. 1 Data flow from satellite to UK data
centre
SDO
4
Data Centre

• Data Centres will allow public access to AIA and
  HMI data.
• Main data centre Stanford University /
  Lockheed Martin
• Online disc cache of science products (line of
  sight magnetograms, images of tracked regions of
  interest, etc)
• Tape archive of low level products (AIA and HMI
  filtergrams)
• Facility to generate high level products
  on-the-fly
• UK data centre currently investigating ATLAS at
  RAL
• Online disc cache of UK requested science
  products
• Rolling 60 day cache of AIA images
• True cache of requested HMI products and older
  AIA data

5
Data Centre with the Grid

• US Data Centre access through VSO, Virtual
  Solar Observatory
• UK Data Centre access through AstroGrid and
  other IVOA compliant virtual observatories
• UK requests will first check UK disc cache for
  data in export format.
• If data is not present, request will be routed
  through an AstroGrid to VSO interface. User will
  get a copy, UK disc cache will get a copy
• Why cant UK users just use VSO?
• They can. However, going through AstroGrid has
  benefits for both the community and individual
  users
• UK cache of request will improve data request
  speed for event-driven science.
• Individuals can take advantage of AstroGrids
  facilities for workflows, access to multiple data
  archives, and algorithmic tool usage.

6
Algorithms

• Three algorithm categories under UK development
• Image and feature recognition (MSSL RAL)
• Global helioseismology
• Local helioseismology

Coronal loop recognition Non-linear magnetic
field extrapolation Helicity computation
Differential Emission Measure (DEM) computation
Small event detection Coronal Mass Ejection
(CME) dimming region recognition
Mode asymmetry analysis Mode frequency
analysis
Perturbation map generation Subsurface flow
analysis Local helioseismology inversion
7
Algorithms with the Grid

• Three methods of algorithm distribution
• Installation at JSOC pipeline in Stanford
  University
• Distribution of IDL wrapped modules through
  SolarSoft
• Deployment of AstroGrid web services

• Minimize data transfer before processing.
• Modules can be designated as automatic or
  on-the-fly during pipeline execution
• Possible access to approved UK users through
  AstroGrid CEA module on top of pipeline.

• Provides access to the algorithm code through
  SolarSoft as used by many in the UK solar
  community.

• Host algorithms on UK machine(s) with AstroGrid
  CEA web service interface
• Allows integration of algorithms with AstroGrid
  workflow to all UK users

8
Visualization

• Three visualization techniques
• Development of quicklook products
• On-the-fly movie and gallery generation
• Zoom and pan streaming tool

• Extract thumbnail images and basic metadata from
  science products
• Generate catalogues from feature recognition
  algorithms (coronal loops, small events, CME
  dimming regions, etc.)

• Generate thumbnail image gallery from
  user-delineated input parameters
• Generate movies from user-delineated input
  parameters

• Open line-of-sight magnetogram or AIA image in
  GUI
• User can zoom and pan in space, time, and
  wavelength / magnetic field.

9
Visualization and the Grid

• Catalogues will be searchable through AstroGrid
  DSA modules.
• Gallery and movie GUIs will be accessible through
  a web browser.
• The Streaming tool might be tricky.
• GUI will be web-based
• Full resolution data products will be streamed in
  multiple resolutions from the cache nearest to
  the user.
• When the user pans or zooms to a product not
  located in local cache, streaming should switch
  to remote cache where product is available. Or
  should this generate a request to transfer the
  product to the local cache?

10
Phase B Work

• Data Centres
• Algorithms
• Visualization

• Deploy AstroGrid DSA and CEA modules for a UK
  disc cache
• Code request mechanism for obtaining export data
  from US data centre
• Test network latency between US and UK
• Liaise with developers at Stanford regarding
  test implementation of AstroGrid modules

• Write and test 11 solar algorithms
• Deploy AstroGrid CEA modules for both UK and US
  hosting machines
• Liasie with Stanford University to experiment
  with using the AstroGrid workflow system in the
  JSOC pipeline

• Liaise with Stanford University to prototype
  streaming tool
• Set up catalogues with AstroGrid DSA module
• Develop GUIs and code for gallery and movie
  generation

11
Thanks
Thanks for your attention. Any questions? eSDO
Project Team
MSSL Elizabeth Auden Len Culhane Mike Smith
Lidia van-Driel Gesztelyi
RAL Andrzej Fludra Richard Harrison Vito
Graffagnino
University of Birmingham Bill Chaplin Yvonne
Elsworth Thierry Toutain
University of Sheffield Michael Thompson Sergei
Zharkov