Combined use of AMDA, CAA, QSAS and CL based on timetable exchange'

1
Combined use of AMDA, CAA, QSAS and CL based on
time-table exchange.
C. Jacquey, V. Génot, E. Budnik, M. Bouchemit, M.
Gangloff, CDPP/CESR B. Cecconi, CDPP/LESIA C.
Perry, CAA team T. Allen, A. Rochel, QSAS
team E. Penou, J.A. Sauvaud, CESR

• This talk is related to
• Poster (Jacquey et al.) this evening
• Talks on CAA (C. Perry), QSAS (T. Allen) and CL
  (E. Penou)

Cluster Workshop, Tenerife, March 12, 2008
2
Some aspects of the studies of space physics

• We need to analyse measurements obtained by
  several observatories
• We need to analyse multi-instrument data
• We need to compare the observations to simulation
  results
• We need to extract and treat a lot of data coming
  from multiple resources ? This takes time
  and energy
• Dynamical processes ?The time is the very first
  parameter
• The studies consists of analysis/characterisation
  of events treated in case studies or in
  statistical studies
• Creating Time-Tables (or event list) is a common
  step of the work

3
Time-Tables (Event Lists)

• Time-Table produced by the users
• Time-Tables coming from statistical studies (e.g.
  magnetopause crossing lists) sometimes made
  available to the community (e.g. the Freys
  substorm list)
• Time-Table coming from search tool based on
  orbitographic or conjonction criteria (e.g. the
  Query tool of the SPDF/SSCWeb)
• Time-Table coming from search tool based on
  mathematical criteria applied to the content of
  the data (e.g. AMDA of CDPP)
• Time-Table coming from visual search tool (e.g.
  AMDA of CDPP)

Exchanging Time-Tables between data centres,
services and tools? ? saving time and energy.
Increase of the scientific return.
4
AMDA
External data
Automated Mutiple Dataset Analysis
Parameter editor
Visualisation editor

• Web based service
• Transparent access to data
• AMDA local database (CLUSTER, ACE, THEMIS,
  GEOTAIL, WIND, , ISEE)
• External database (CDAWeb, MAPSKP, THEMIS/CESR,
  )
• Produces and exploits time-tables

Download data
Conditional search
Visual search
Time-Table manager
AMDA DOES NOT WORK WITH InternetExplorer and
Safari AMDA is public (http//cdpp-amda.cesr.fr)
5
Resources data centres, services, tools
CDPP
Mirror THEMIS database
CDPP database
AMDA ? Time-Tables
QSAS
CDAWeb
CL
6
Science Casestudy and characterisation of the
local properties of the cross-tail current

• Based on CLUSTER data when it encircles the
  neutral sheet.
• Current density, profile, gradient
• Thickness, location, motion
• Distribution function (i and e-), flows,
• Turbulence/wave level and spectra
• Influence factor
• Solar Wind pressure
• IMF
• Geomagnetic activity
• State of the ionosphere.
• Oxygen abundance
• Ring current intensity

7
Use Case the main steps
Selection of the periods of NS
encircling condition
Access and extraction of modelisation
or simulation data (Tsyganenko, CCMC, )
Access and extraction of ground (AE, AL, SYM,
DST) data
Formatting
Generic tools
Access and extraction of CLUSTER data
Access and extraction of CLUSTER data
Data merging
Specific tools
Access and extraction of ACE/WIND/GEOTAIL Interpla
netary data
Time shifting
User tools
Results Catalogues Maps Models
Basic treatments
CAA
8
Creation of time-tables on AMDA/CDPP

• TT1, time-table listing the periods when the
  conditions are fulfilled
• TT2, extended time-table

9
Conditional search
10
Visualisation
11
Visual search
12
Managing Time-Tables
13
Parameter editor
14
Data dowload
15
First step producing sub-databases corresponding
to the time-tables
CAA Validated high resolution CLUSTER data

• AMDA
• Automated search
• Visual search
• ? Time-Tables

Time-Tables
Data, parameters
Data, parameters
Standardised sub-databases
User tools
QSAS
CL
16
How to do that?

• Standardisation of the time-tables
• Standardisation of the organisation of the
  sub-databases to be produced
• Interface implementation on CAA for reading the
  time-table and producing sub-databases
• Interface implementation on the tools QSAS and CL
• Standardisation of the time-table description

17
Step 2 catalogues
CAA, CDAweb, CDPP or other data centres
QSAS, CL, AMDA or other tools
AMDA, SSCWeb or other services
18
Future Archiving the catalogues for availability
to the community
Catalogue T1, T2, P1, , PN
USER
Sub-database
QSAS or CL
Catalogue T1, T2, P1, , PN, Q1, ,QN
19
Conclusion

• Exchange of standardised Time-Tables between data
  centres, services and tools should allow the
  researcher
• - to save large amount of time and energy
• - to exploit more extended data amount
• - to increase the scientific return of the data
• First step extraction and exploitation of
  sub-databases corresponding to the Time-Tables
• Second step creation and exploitation of
  catalogues, similarly to astronomy and solar
  physics
• This project starts with CAA, QSAS, CL and AMDA,
  but of course, other new participant is welcome
• The specifications of this project need the input
  of the potential users
• ? Poster this evening in order to collect
  suggestions and ideas.

AMDA contact amda_at_cesr.fr Stable
version http//cdpp-amda.cesr.fr Test
version http//iapetus.cesr.fr/AMDA_PRETEST/
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Use cases

• Use case simulation of studies which could be
  performed by the researchers in order to
• identify and analyse their needs
• specify the user requirements
• Imagine, define and specify the system to be
  developed
• The developers of the system need the input from
  the community

• Obvious requirement standardisation
• Standardisation of the data format
• CEF, CDF, NetCDF, (FITS)
• But beyond that, standard way for representing
  data of different classes (particle , waves,
  Time-Tables, )
• Standardisation of the data description
• ? SPASE (Space Physics Archive Search and
  Extract,
• http//www.spase-group.org/ )

25
Step 2 sub-database extraction

• On CAA, CDAweb, DARTS, CDPP, AMDA,

• Requirements
• Standard format for time-tables
• Interface for reading and managing the
  time-tables
• User interface for selecting data
• Optionally, basic on the fly treatment (e.g. data
  merging, SW propagation shifting, )
• Standardised management of the multiple file
  output
• Filename
• Descriptor
• Product recovering

26
download
27
Pi-Pj/(Xi-Xj)
28
Step 3 plot production on CL, QSAS, AMDA

• INPUT
• ? time table
• ? corresponding sub-database
• Manipulation of the time-table
• User edited plot
• Launch in batch mode
• Requirements
• ? Standardisation (i) of the time-table and (ii)
  of the data
• ? Plot editor
• ? Batch mode
• ? Standardised management of the plot files
• ? Filename
• ? Descriptor
• ? Product recovering

29
AMDA
Automated Mutiple Dataset Analysis
Created by the CDPP http//cdpp.cesr.fr/
Stable version http//cdpp-amda.cesr.fr/ Testin
g version http//iapetus.cesr.fr/AMDA_PRETEST/ C
ontact amda_at_cesr.fr
30
Example event search
Old fashion paper search
ISEE MAG
IMP8 PLA
ISEE- electron
AE, AL
IMP8 MAG
IMP-8
AE, AL
ISEE-data
Current fashion web
CDAWeb SPDF
UCLA
CAA
CDPP
And others
Event search takes time and energy
31
AMDA the goals
Automated Mutiple Dataset Analysis
Multiple Dataset the study of the (multi-scale)
dynamics of plasma objects requires to perform
the integrated analysis of multi-point and
multi-instrument data

• Automated (or semi-automated) Analysis help for
• Search, characterisation, classification of
  events
• Extensive exploitation of the vast database
  (statistics)
• historical studies (example through solar
  cycles)
• Building catalogues
• Building virtual constellations
• Time-Table (EventList) production

32
AMDA, fact sheet
Built in IDL, C, FORTRAN, Javascript Works
on standardised and simple data, NetCDF
Includes models (Tsyganenko, Shock, MP, NS, )
Uses external data (now from CDAWeb and soon from
MAPSKP, IWF, THEMIS, ) Forum for user
feedback Twiki for information (development,
bugs, ) exchange Already in use for
scientific studies AMDA DOES NOT WORK WITH
InternetExplorer and Safari AMDA is public
33
Functionalities

• Automated access to data
• Access to and on-the-flight ingestion of external
  data (test version only)
• Data merging
• Data download
• Data visualisation
• User edited parameter computation on the data
• Automated conditionnal search
• Visual search

34
AMDA/External Data
MAPSKP
AMDA
CDPP
System
Local Database
CDAWeb
others
35
Needed development on AMDA

• Temporal functions (mean, regression, dispersion
  estimator, )
• User pluggins
• Continuous SW propagation shifting
• Scatter plots, map plots,
• Your suggestion
• Your suggestion
• Your suggestion
• Your suggestion
• .

36
Conclusion and questions

• CAA, QSAS, CL and CDPP teams are starting a
  collaboration for providing to the researchers
  ways to save time and energy for performing their
  studies
• Other services and database are invited to join
  this effort
• We need input, comments, suggestions from the
  potential users.