Global Auroral Imaging Access

1
Global Auroral Imaging Access
2
Why might others interested in this?
3
Why are we interested in this?
THEMIS 6 ASIs Now, 20 by October
2006
NORSTAR 7 ASIs now, 12 by 2006
- 4 MSPs - 13 single beam
riometers - Weatherwax IRIS by
2005
4
Why are we interested in this?
By 2007

• we will be running 35 ASIs
• we will be producing 10TB of image data per
  year
• we are operating 13 riometers
• we are operating 4 MSPs
• we are a partner in other riometer, MSP, and
  ASI projects
• we have past optical data sets- Wilbur, POCA,
  ISIS, Viking,
• Freja, Interball, etc.

There is no existing way available for us to
manage our data We are going to solve our
problem We want to do this in a way that is
scalable so in principle our solution could be
the kernel of a solution to the global
problem. We have an unfunded mandate we can
do whatever we want
5
Horizontal Vs. Vertical Observatories

CGSM
MIRACLE
THEMIS
Oulu
SPEARS
BAS
AGI
X X X X X X
X X X
X X X X
X X X X X X
X
X X X
X X
X
ASI
MSP
RIO
MAG
D-SONDE
HF RADAR
VHF RADAR
VHF

6
Horizontal Vs. Vertical Observatories

CGSM
MIRACLE
THEMIS
Oulu
SPEARS
BAS
AGI
X X X X X X
X X X
X X X X
X X X X X X
X
X X X
X X
X
ASI
MSP
RIO
MAG
D-SONDE
HF RADAR
Vertical Observatory
VHF RADAR
VHF

7
Horizontal Vs. Vertical Observatories

CGSM
MIRACLE
THEMIS
Oulu
SPEARS
BAS
AGI
X X X X X X
X X X
X X X X
X X X X X X
X
X X X
X X
X
ASI
MSP
Horizontal Observatory
RIO
MAG
D-SONDE
HF RADAR
SuperDARN
VHF RADAR
VHF

8
Horizontal Vs. Vertical Observatories

CGSM
MIRACLE
THEMIS
Oulu
SPEARS
BAS
AGI
X X X X X X
X X X
X X X X
X X X X X X
X
X X X
X X
X
ASI
MSP
RIO
VGMO
MAG
D-SONDE
HF RADAR
SuperDARN
VHF RADAR
VHF

9
Horizontal Vs. Vertical Observatories

CGSM
MIRACLE
THEMIS
Oulu
SPEARS
BAS
AGI
X X X X X X
X X X
X X X X
X X X X X X
X
X X X
X X
X
ASI
GAIA
MSP
RIO
VGMO
MAG
D-SONDE
HF RADAR
SuperDARN
VHF RADAR
VHF

10
NORSTAR MIRACLE THEMIS AGI UNIS SPEARS DMI IRIS
PENGUIn Oulu BAS Sonde
CARISMA CANMOS MACCS AGI IGPP MEASURE MERIDIAN USG
S SAMBA IMAGE
GAIA
VGMO
Super VO
Clients
Saskatoon Fairbanks APL Leicester Latrobe Durban
BAS
SuperDARN
11
What is GAIA ???
Client
Public Interface

• Get Summary URL
• Get Data
• Get Data Availability

GET INFO
SRI
MIRACLE
Others?
NORSTAR
AGI
THEMIS
SPEARS
12
What is GAIA ???
Client
Public Interface
GAIA
GET INFO
SRI
MIRACLE
Others?
NORSTAR
AGI
THEMIS
SPEARS
GAIA will be a public interface (set of tools),
to browse summary data, data availability, and
retrieve data from a worldwide network of ASIs,
MSPs, and Riometers.
13

• GAIA has to do the following ----
• Create a public interface for browsing summary
  data / data availability from multiple
  instruments
• Have the ability to interface with remote data
  providers and serve up summary data through a
  common interface (transparent)
• Allow clients to query information about data
  availability.
• Allow clients to request and retrieve standard
  imaging data.

14

• GAIA has to do the following ----
• Create a public interface for browsing summary
  data / data availability from multiple
  instruments
• Have the ability to interface with remote data
  providers and serve up summary data through a
  common interface (transparent)
• Allow clients to query information about data
  availability.
• Allow clients to request and retrieve standard
  imaging data.

1. WebServices tools talking to tools
2. XML structured data transfer, designed to be a
   self-describing data language - overkill
3. SOAP communication protocol based on XML,
   provides a way to communicate between
   applications running on different servers/systems
   
4. HTTP/CGI-GET - widely used and simple, easy to
   implement with existing servers, allows rapid
   development -

Options
15

• GAIA has to do the following ----
• Create a public interface for browsing summary
  data / data availability from multiple
  instruments
• Have the ability to interface with remote data
  providers and serve up summary data through a
  common interface (transparent)
• Allow clients to query information about data
  availability.
• Allow clients to request and retrieve standard
  imaging data.

1. WebServices tools talking to tools
2. XML structured data transfer, designed to be a
   self-describing data language - overkill
3. SOAP communication protocol based on XML,
   provides a way to communicate between
   applications running on different servers/systems
   
4. HTTP/CGI-GET - widely used and simple, easy to
   implement with existing servers, allows rapid
   development -

Options
16
How it Will Work
Interface (scripts)
GAIA Database Management System (DBMS)
Data Catalogue
Meta Data
Summary Data
GAIA Data Producers
Data Sets
Summary Data
Full Data
17
How it Will Work
Interface (scripts)
GAIA Database Management System (DBMS)
Data Catalogue
Meta Data
Summary Data
GAIA Data Producers
We do not need a centralized data archive.
Using a DBMS we can formulate local or remote
URLs to serve data through a common interface
transparently.
Data Sets
Summary Data
Full Data
18
How it Will Work
GAIA Database Management System (DBMS)
Data Catalogue
Meta Data
Summary Data
All that we need is the ability to formulate a
URL location of Summary data.
19
How it Will Work
GAIA Database Management System (DBMS)
Data Catalogue
Meta Data
Summary Data
keograms Mosaics Images time series
station info instrument mode content summary data
data availability data location data policy meta
data summary data
This is the only element of GAIA that must reside
somewhere
20
How it Will Work
GAIA Database Management System (DBMS)
Example

Query - all imagers with Project
THEMIS allow the user to stack THEMIS
keograms Query - all imagers within XXkm of
Gillam allow users to view data relevant to
different scale sizes
scalability
Allows us to put any kind of interface on the
front end with amazing functionality
21
How it Will Work
Client
Public Interface
The location of the summary data is transparent
URL info
GAIA
GET INFO
SRI
MIRACLE
Others?
NORSTAR
AGI
THEMIS
SPEARS
22
How it Will Work
Client
Public Interface
GAIA
GET INFO
SRI
MIRACLE
Others?
NORSTAR
AGI
THEMIS
SPEARS
Clients
23
What GAIA Should Do
GAIA V1.0
Start with a Summary Browser
24
What GAIA Should Do
GAIA V1.0
Summary Data 1 minute thumbnails
24 hour keograms 2
week summary plots
25
What GAIA Should Do
GAIA V1.0

• Three Interfaces
• Keogram Driven
• Two Week Summary
• All Sites One Day

Choose a time by clicking on keogram or using the
forward/back arrowsinterface will display five
thumbnails centred about the selected
UT Interface will respond to a location
window.the user may change the instrument/site
dynamically, while retaining all date time
information
Imager Information Display
26
What GAIA Should Do
GAIA V1.0

• Three Interfaces
• Keogram Driven
• Two Week Summary
• All Sites One Day

Two week interval click on any given day to move
to the keogram driven interface for that
day Again, responds to the location window.
27
What GAIA Should Do
GAIA V1.0

• Three Interfaces
• Keogram Driven
• Two Week Summary
• All Sites One Day

Query the GAIA database. Output all one day
keograms for that day (seperated by filter) and
stack them on the screen. Dynamic filter control.
Flip between energy ranges / intruments.
28
What GAIA Should Do
GAIA V1.0
Have the ability to request summary data. The
query is passed to the DBMS and all available
summary data is collected and placed in a
location for pickup by the user.
Have the ability to query data availability The
query is passed to the DBMS / Data Catalogue,
returning the availability of the request and/or
the contact information of the GAIA Data Provider.
29
Work Necessary For Your Project to Be in GAIA
GAIA V1.0
Image (ASI Riometer) Components Summary
Browser

• Need to create a set of images compatible with
  the GAIA browser
• 100x100px thumbnails (grayscale)
• 720x200 daily keograms/riograms
• (Note GAIA creates two week summary
  plots)

Database Information

1. Directory tree structure
2. Filenaming conventions
3. Imager Information
4. Lat / Lon
5. Filter
6. Exposure

Ability to formulate URL
Ability to search based on filter, location,
content, etc.
30
Notes to Data Providers
GAIA V1.0
We do not need to host summary data
GAIA will link into summary data and serve it in
a common interface
We simply need the ability to locate Summary
Images / query availability
31
GAIA Granularity
32
GAIA Granularity
This level of summary reduces volume by a bit
more than 2 orders of magnitude. There are not
actually a lot of options here.
33
GAIA Granularity
34
GAIA Granularity
35
GAIA Milestones
Summary Browser

• three interfaces
• keogram driven
• two week summary
• all sites (single day)
• query data availability
• more interfaces
• dynamic plots
• mosaics
• real-time
• retrieve (CDF) data on demand
• mirror sites
• Orientation
• Mapping
• Cloud Cover
• Classification
• Mosaicing
• Keogram Plotter
• Ephemeris

V1.0
Sept. 1, 2005
Data Access
Summary Browser
V2.0
Mar. 1, 2006
Data Access
Data Access
Software Package
ongoing
36
GAIA Working Group

• Emma Spanswick
• Mikko Syrjäsuo
• Brian Jackel
• Steve Marple
• Eric Donovan
• Stephen Mende
• Farideh Honary
• Kirsti Kauristie
• Rick Doe
• Dirk Lummerzheim
• Hans Stenbaek-Nelsen
• Al Weatherwax
• Thayil Jayachandran
• Andrew Kavanagh
• Kazuo shiokawa

37
(No Transcript)
38
Learn by doing!