Development of new tools and interoperability at CDPP'

1
Development of new tools and interoperability at
CDPP.
Christian Jacquey
The CDPP (Centre de Données de Physique des
Plasmas) is the french national centre of natural
plasma data.
Graz, IWF, Nov. 23, 2007
2

• Origin and missions of the CDPP
• First phase of CDPP (1998-2005)
• Present phase (2005-)
• Context
• Scientific targets
• AMDA (Automated Mutli-Dataset Analysis)
• Scientific animation
• Potential collaborations

3
Origine and missions of the CDPP
The CDPP was created in 1998 by both CNES and
CNRS. The CDPP is hosted at CESR, Toulouse
1st mission long term archive of plasma data
obtained with experiment realised with french
participation, making them available to the
community 2nd mission providing added value on
the archived data ? services, tools 3rd
mission participation to Virtual Observatory
projects 4th mission scientifical animation
(workshop, )
4
First phase 1998 - 2005

• Archiving more than 250 datasets, sometimes
  saving almost lost ones
• (GEOS 1,2, ISEE 1, 2, 3, ARCAD, WIND, ULYSSES,
  INTERBALL, CLUSTER,)
• But
• Data available only in the native format
•  Isolated  data sets
• Very difficult to develop tools and services
• Small number of users
• Implication in the interoperability development,
  founder member of the SPASE consortium

5
Second phase 2005 -

• Technological context
• The volume, the diversity and the complexity of
  the data are increasing more and more
• The capacities of the machines and the networks
  are increasing
• The interoperability is developing
• Scientific context
• THEMIS launched in February 2007, simultaneously
  flying with CLUSTER, DSTAR, GEOTAIL, ACE, WIND,
  and others
• STEREO launched in October 2006, simultaneously
  flying with ULYSSES, ACE, WIND
• Planetary missions are flying VEX, MEX,
  MESSENGER, CASSINI

So many data ? fantastic scientific
opportunities ? but need of infrastructures for
helping researchers to take advantage of
them
6
Scientific targets

• Heliophysics, Sun-Earth relationships
• Magnetospheric physics
• Planetology

7
Heliophysics, Sun-Earth relationships

• Origin, propagation, characterisation,
  classification of the solar/interplanetary
  disturbances (CME, CIR, interplanetary shocks,
  clouds, filaments, )
• Identification of the mechanism and sites of the
  acceleration of the solar energetic particles and
  their propagation
• Coupling to the Earth, geoeffectivness, level of
  disturbances in the ionosphere and the internal
  magnetosphere
• Modelling the 4D-heliosphere
• Predicting the heliospheric conditions space
  weather
• Predicting their impact on the Earth environment

8
Large scale constellation
9
The multi-scale heliospheric constellation
Continuous solar observations
SOHO, STEREO, HINODE, RHESSI, Ground observatories
Constellation of probes distributed at large
scale
Heliospheric probes STEREO-A/B, ULYSSES,
VOYAGER Planetary probes MESSENGER, VEX, MEX,
MGS, CASSINI
Constellation of probes distributed at medium
scale around the Earth orbit
ACE, WIND, THEMIS, GOES, GEOTAIL, CLUSTER, LANL
Two sub-constellations in small scale cluster
configuration
THEMIS, CLUSTER,
detailed earth-ionosphere data
astronomical observations (aurora)
10
General functions
X-ray GOES
Links to simulation
FALSE DATA
SOL/tools
PLAS tools
Time tools
SOL/Workspace/Composer
PLAS/Workspace/Composer
11
Magnetospheric physics

• Large-scale process at the origin of substorm
• Instabilities
• Multi-scale dynamics of the magnetopause versus
  solar wind or magnetosheath conditions
• Particle acceleration (shock, magnetotail, cusp,
  )
• Turbulence
• (so many interesting topics)

12
The magnetospheric constellation
13
Planetology

• Comparative study
• Dynamics of the planetary ionised environment
  versus solar wind conditions
• Plasma interaction with moons, rings, dust,
  neutrals
• Magnetosphere without ionosphere (Mercury)
• Multi-species plasmas
• Plasmas in small structures (comets, L lt RL)

14
Example of comparative study
H
O
VENUS
MARS
Ferrier et al.
15
Basic requirements

• Easy access to many datasets from various origins
• Standardisation of the data for making them
  usable together
• Data in physical values
• Tools for exploring the content of data
• Taking into account the present context, the
  development data centres are one of the keys for
  getting a fruitful return on the experimental
  investment

16
Toward a new CDPP

• Rebuilding the system
• Completing the data collections
• Making  hot  data available
• Standardisation of the data to make them usable
  by generic tools
• Developing tools (AMDA)
• Developing concrete application of
  interoperability, in collaboration with other
  data centres or data/service providers

17
Hot data at CDPP

• CLUSTER Prime Parameters
• High-res data for CLUSTER/WHISPER, STAFF, CIS
• The CDPP hosts a full mirror THEMIS database
  (space and ground data)
• DEMETER
• STEREO/SWAVES, IMPACT
• VEX, MEX ASPERA
• But also older data, useful for very large
  statistical analysis
• And remote access to the whole CDAWeb database,
  and soon, the CASSINI/MAPSKP and the VEX/MAG data

18
AMDA
Automated Mutli-Dataset Analysis
19
Example event search
Old fashion paper search
ISEE MAG
IMP8 PLA
ISEE- electron
AE, AL
IMP8 MAG
IMP-8
AE, AL
ISEE-data
Current fashion web
CDAWeb SPDF
UCLA
CAA
CDPP
And others
Event search takes time and energy
20
AMDA the goals
Automated Mutiple Dataset Analysis
Multiple Dataset the study of the (multi-scale)
dynamics of plasma objects requires to perform
the integrated analysis of multi-point and
multi-instrument data

• Automated (or semi-automated) Analysis help for
• Search, characterisation, classification of
  events
• Extensive exploitation of the vast database
  (statistics)
• historical studies (example through solar
  cycles)
• Building catalogues
• Building virtual constellations
• Time-Table (EventList) production

21
AMDA, fact sheet
Built in IDL, C, FORTRAN, Javascript Works
on standardised and simple data, NetCDF
Includes models (Tsyganenko, Shock, MP, NS, )
Uses external data (now from CDAWeb and soon from
MAPSKP, IWF, ) Forum for user feedback
Twiki for information (development, bugs, )
exchange Already in use for scientific
studies AMDA DOES NOT WORK WITH InternetExplorer
and Safari AMDA is public
22
Functionalities

• Automated access to data
• Access to and on-the-flight ingestion of external
  data (test version only)
• Data merging
• Data download
• Data visualisation
• User edited parameter computation on the data
• Automated conditionnal search
• Visual search

23
AMDA, DEMO
24
(No Transcript)
25
Conditionnal search
B
B

• STEP 1, on AMDA search
• Test X1lt -10 Re
• BX1 BX2 BX3 BX4 gt 0
• min(BX1 BX2 BX3 BX4) lt 0
• max(BX1 BX2 BX3 BX4) gt 0
• Time-Table

26
AMDA/External Data
MAPSKP
AMDA
CDPP
System
Local Database
CDAWeb
others
27
Scientific animation

• Organisation of workshops
• Hosting exchange pages (twiki)
• Hosting researchers for extended use of AMDA

28
Collaborations
Targeting data or service exchanges,
interoperability, Virtual Observatory
construction ? Developing more and more data
access/processing/analysis facilities for serving
the community
Virtual Observatory? Example a VO for
heliophysics CDPP participates to the FP7
proposal HELIO
29
Necessity to access to many data and tools
Solar data
Heliospheric data
CDPP
MEDOC
BASS2000
NSSDC
Planetary data
PDS
NSSDC
PSA
CDPP
USER
NSSDC
CDPP
CAA
Iono/Magnetospheric data
30
A virtual observatory
Solar data
Heliospheric data
CDPP
MEDOC
BASS2000
NSSDC
Planetary data
PDS
NSSDC
PSA
CDPP
VO
USER
NSSDC
CDPP
CAA
Iono/Magnetospheric data
31
Baby magnetospheric VObased on Time-Table
exchange
Collaboration discussed with CESR, CAA, GAIA and
CDAWeb. Already working between AMDA and CL

• Definition of a standard format for time-table
  (likely VO-Table)
• Implementation of interface for Time-Table
  exchange

32
Time-Table a simplified use case
Science goal study based on CLUSTER of the
cross-tail current in the neutral sheet in
relation with interplanetary and geomagnetic
activity conditions

• STEP 1, on AMDA search
• Test X1lt -10 Re
• BX1 BX2 BX3 BX4 gt 0
• min(BX1 BX2 BX3 BX4) lt 0
• max(BX1 BX2 BX3 BX4) gt 0
• Time-Table 1
• STEP 2 Edition of Time-Table1
• Adding 2 hours before and after the time
  intervals of Time-Table 1
• ?Time-Table 2
• STEP 3, on CAA Sending the Time-Table2 and
  extracting a sub-database
• STEP 4, on GAIA Sending the Time-Table2 and
  extracting a sub-database
• STEP 5, on CDAWeb Sending the Time-Table2 and
  extracting a sub-database
• STEP 7, on CL Sending Time-Table1 and looking
  at the distribution function
• Etc

33
Collaboration CDPP/IWF on planetology

• Final goal construction of a VO for planetology
• VERY CHALLENGING strong heterogeneity of
  disciplines, cultures, types of data
• Short term
• Demonstrator based on AMDA using local ASPERA
  data and distant data (VEX-MAG at IWF, MAPSKP at
  CESR)

34
Conclusion
The CDPP is in strong evolution The CDPP
develops tools for serving the community. These
tools are free and public. The CDPP is opened
for hosting workshops or visiting
researchers The CDPP is volunteer to participate
or stimulate collaboration for developing new
services, facilities or infrastructures helping
the progresses of the community.