CSTS-File Transfer service discussions (2)

1
CSTS-File Transfer service discussions (2)

• CNES position

2

• File Content Description
• Define manifest files identifying
• File content (to the level that the recipient
  needs to know)
• File format?
• Processing Instructions
• Combine with content description?
• Pull Model, Push Model, or both?
• Mouy Francois-Xavier No needs identified yet
  for push model (but still a little bit of work to
  do)
• Subscription and automatic push? Mouy
  Francois-Xavier for push model
• Subscription and notification of new data? Mouy
  Francois-Xavier how send notifications if
  provider not bind. Bind return could notify a
  list of avaliable data
• File Management Services (see dedicated slide)
• File Directory / Catalogue Servcies
• File Management Mouy Francois-Xavier for
  push model
• Create / Delete
• Move
• Local copy

3

• Security (see dedicated slide)
• Off the shelf FT protocols support security but
  must be configured and depending on features used
  may need supporting infrastructure
• Firewall issues
• Access Control
• At what level? (mission, individual accounts, )
  Mouy Francois-Xavier Mission level seem to be
  the right level
• Access Control Groups
• More complex schemes
• What level of privacy / access control is needed?
  Mouy Francois-Xavier We need to be able to
  provide this kind of features before security
  person's asks
• E.g. can user X see that data are stored for user
  Y? Mouy Francois-Xavier he shouldn't -)
• Key management issues
• Others?

4

• Reading Martin initial thoughts.
• Protocol choices are wide. If we use one or other
  protocol, it comes with all this complexity
  (advantages, drawbacks).
• Some features of standards COTS are not useful in
  our context, but you have to tune and secure
  those features anyway.
• Most of the difficulty come with the push model
  (security, behavior), the pull model is very near
  the existing offline model.
• Initial CNES position
• Firstly, we thought we only need the pull model
  which is quite simple than the push model.
• Furthermore, File transfer function is already
  done by other ways. (ftp servers in parallel)
• Therefore, why redo it ?

5

• However, Experience show us that deploying a file
  transfer feature with common COTS is not quite
  easy that it is on paper (e.g. with ftp passive
  form, active , dynamic port ranges, firewalls,
  user management, ).
• Using CSTS could simplify the integration because
  the network security aspects are already done. A
  part of the reporting too (notifications).
• CNES think CSTS could be a good  Secured Pipe 
  to the Ground center.
• A binded instance is maintained (Heartbeat)
• The Start/Stop mechanisms could be useful to
  drive providers behavior
• We are able to use CSTS services via a high level
  of firewalls system

6
Candidate standards (from initial thoughts)

• File Transfer Standards
• FTP, FTPS, SFTP (SSH based FT)
• HTTP / HTTPS ?
• Others?
• CCSDS Standards
• XFDU ?
• Others?

7
XFDU (xml formatted data unit)

• GOAL organizing transferred data in an Open
  Archive environment (OAIS)

A container (zip file) with a descripition of its
content in a manifest file. The logical
description points to the physical content
which can be either local or distant
8
XFDU description

• The Information Package Map contains the logical
  view of the package. Its a hierarchical xml tree
  representing the content of the package. Each
  leaf of the tree is a Content Unit and can be
  referred to from the other parts of the package
  (i.e. information is linked by the Content Unit
  reference ID).
• The Data Object Section contains all the physical
  information needed to get the file objects out.
• The Metadata Section records all of the metadata
  for all items in the package.
• The Behavior Section associates executable code
  with the content of the package.

9
XFDU manifest structure
The structure map is a hierarchical view of
logical references which are pointing to
physical data, metadata and specific behaviors.
10
XFDU benefits

• Unique Identification of the data ( ContentUnit
  ID)
• Interoperability by using an XML schema
  description
• Hierarchical structure of the information
• Data physical access (DataObject) separated from
  the logical concepts (ContentUnit)
• Metadata associated with the ContentUnit
• Specific processes associated with the
  ContentUnit (BehaviorObject)

11
XFDU drawbacks

• XFDU developed for approaching needs but not
  totally the sames (multi-parts, files, cdrom)
• Few implementations due to standard complexity.
• NASA XFDU library, ESA SAFEs archive for
  ENVISAT data, CNESs PAIMAS
• Behavior function not really validated

12
CSTS-XFDU conclusion

• The data organization has been studied earlier by
  the MOIMS/IPR group. XFDU is an answer to csts
  concerns with the following conditions
• Needs of an XFDU  light  to answer the
  requirements.
• Needs to study the behavior statement.
• .

13
CSTS-XFDU conclusion

• XFDU (simplified one) Could be a good system to
  integrate the file transfer feature.
• The behavior side of XFDU could be a response to
  File transfer and more
• CNES thinks it could be a possible cooperation
  and some people are interested to study those
  aspects

14
(No Transcript)
15
Some use cases at CNES

• Distant management of stations
• Management by xfdu of TM archived in station
• TM playback via CSTS-offline service
• Inter-facilities scheduler
• Distant scripts used by the scheduler managed by
  xfdu
• Launch via services CSTS
• Re-use of CNES well-known scheduler GUI

16
Lets take an example.

• XFDU over CSTS for playing recorded telemetry

17
Telemetry transfert and play

• A facility wants to transfert simulated telemetry
  to a station and to be able to play it back
  later.
• The user build an xfdu package (X1) with the TM
  file and two scripts (play and rewind)
• All the configuration and security concerns to be
  solved by the management layer.

18
STEP 1CONNEXION
user
provider
CSTS FT
CSTS FT
bind
XFDU X1
TM 1
PLAY
REW
19
STEP 2XFDU TRANSFERT
user
provider
CSTS FT
CSTS FT
XFDU X1
bind
TM 1
START PUT X1
XFDU X1
TM 1
PLAY
REW
PLAY
REW
20
STEP 3XFDU AUTOMATICINSTALL
user
provider
CSTS FT
CSTS FT
XFDU X1
bind
TM 1
START PUT X1
XFDU X1
TM 1
PLAY
INSTALL
REW
PLAY
REW
PLAY
REW
TM 1
21
STEP 4The user lists all available xfdusand
their behaviors
user
provider
CSTS FT
CSTS FT
XFDU X1
bind
TM 1
START PUT X1
XFDU X1
TM 1
PLAY
INSTALL
START LIST
REW
X1
PLAY
START LIST X1
REW
PLAY
PLAY, REW
REW
TM 1
22
STEP 5The user plays the TM1 telemetry back
user
provider
CSTS FT
CSTS FT
XFDU X1
bind
TM 1
START PUT X1
XFDU X1
TM 1
PLAY
INSTALL
START LIST
REW
X1
PLAY
START LIST X1
REW
PLAY
PLAY, REW
REW
START RUN X1 PLAY
TM 1
TM 1