Data centre support for the IGSRT PP

1
Data centre support for the IGS-RT PP
W. Söhne, H. Habrich, G. Weber Federal Agency for
Cartography and Geodesy, Frankfurt am Main,
Germany
2
Outline

• Introduction
• IGS-RT PP
• Data and product file centre
• File types under consideration
• Technical aspects
• Data volume
• Up- and downloading
• Completeness
• Limitations
• Policy aspects
• Upload policy
• Conclusions open questions

3

Introduction

• Storing of highrate files nothing new for IGS
• At CDDIS back to 2001, doy121
• Initiative from the LEO needs
• At IGN
• Others
• Term highrate is a relative one
• Usually means 1 Hz sampling rate
• But higher sampling rates in use depending on the
  application
• Currently 110 stations at CDDIS
• Clear statement from IGS concerning the needs of
  long-term archive still missing

4
IGS-RT PP (1)

• One activity of the IGS-RT PP Data and product
  file centre
• File types under consideration
• Navigational data (broadcast ephemeris)
• Observational data
• highrate RINEX files
• hourly and daily RINEX files
• Derived files (products)
• Orbits or orbit corrections
• Clocks or clock corrections
• Troposphere ionosphere parameters

5
IGS-RT PP (2)

• Why generation of highrate files?
• Currently, considerable number of users are
  feeding near real-time applications
• For future scientific analyses
• What is near real-time?
• Typically hourly applications, with 20-30 minutes
  computation time
• ? two times per hour computation possible
• Why 15 minutes files?
• Reduction of file size compared to an hourly
  highrate file
• Convention

6
IGS-RT PP (3)

• Is that interval small enough? Are there other
  intervals of interest, e.g. 5 minutes files?
• Advantages
• Better distribution of uploading over time
• Other intervals can easily be created/derived
• File naming convention fits
• Disadvantages
• Interval size not known in IGS
• Number of files growing

7
Technical aspects data volume

• Data volume of highrate observational files
• Example BUTE115A45.08O 1 Hz 15 minutes
  GPSGLONASS RINEX file, obs. types 8, 20 SV in
  view 2.4 Mb
• Hatanakacompress 170 Kb
• ? 5.7 Gb per station and year for the compressed
  files
• ? gtgt 500 Gb per year
• Selection of IGS stations for high-rate storing?
• Data volume of product files
• Clocks 1 Hz, 1 hour 2.5 Mb, compressed 170 Kb
  ? 5.7 Gb per solution and year

8
Technical aspects up- and downloading

• Data upload of observational files
• Example 250 stations, 24 files per day (hourly)
  6000 files uploaded 100 Kb ? 0.6 Gb per day
• Example 110 stations, 96 files per day
  (highrate) 10560 files uploaded 170 Kb ? 1.7
  Gb per day
• Critical aspects are
• Is the data centre able to handle all incoming
  files within, e.g., 5 minutes after the full
  hour?If not ? spreading of upload over a certain
  time span?!
• Is the data centre able to handle the parallel
  download requests in peak periods?

9
Technical aspects completeness (1)

• Completeness covers the aspects
• Number of observation types
• Number of epochs
• Number of SVs per epoch
• Ratio observed / predicted number of observations
• Completeness can be affected by
• Outages of single stations data streams
• Outages of the broadcasting system
• Handling of unhealthy SVs
• Limitation of supported observation types

10
Technical aspects limitations (1)

• Limitations cover the following aspects
• Resolution of
• Phase
• Code
• Support of
• GPS L2C, L5
• GLONASS
• Galileo
• SBAS

11
Technical aspects completeness (2)

• Different levels of comparison of original and
  accumulated files possible
• Character by character
• Difficult due to roll-over phase values
• Completeness
• Epochs missing?
• SVs missing?
• Parallel analysis
• Differences between the results

12
Technical aspects completeness (3)
Hourly file BUTE115A.08O (TEQC)
Highrate file BUTE115A00.08O (BNC)
13
Technical aspects limitations (2)
ALME file from TEQC vs. file from BNC (RTCM 2.3)
after RNXSMT (BSW5.0)
14
Technical aspects limitations (3)
WTZR file from TEQC vs. file from BNC (RTCM 3.0)
after RNXSMT (BSW5.0)
15
Technical aspects completeness (4)
Completeness of 711 hourly high-rate RINEX files
from RTCMv3 streams coming in via NTRIP/TCP
Completeness of 233 hourly high-rate RINEX files
from RTCMv2 streams coming in via NTRIP/TCP
Completeness of 24 hourly high-rate RINEX files
from RTIGS streams coming in via udpRelay
16
Technical aspects completeness (5)
17
Policy aspects (1)
Traditional situation transferring hourly
files, daily files, ephemeris files
Station A
Data centre 1
User a
Station B
User b
Mirroring
Station C
Data centre 2
User c
Mirroring
Data centre 3
Station N
User m
? file creation at the station
18
Policy aspects (2)
Possible situation using real-time data streams
for RINEX file creation
Broadcaster 1
Station A
User a
Station B
or
Station C
User b
Data centre 1
Station N
? file creation possible at the data centre, at
the users site, at a third party
19
Conclusions open questions (1)

• Derivation of highrate RINEX files from real-time
  streams suitable tool
• Current interval for highrate files 15 minutes
  is that small enough?
• ? Recommendation 1 Storing of 1 Hz 15 minutes
  files, at least for the long-term archive
• Derivation of daily files from streams instead of
  ftp transfer, at least for new or proposed
  stations?
• RT-IGS PP CfP found five candidates (BKG, CDDIS,
  GA, KASI, Univ. of Padova)

20
Conclusions open questions (2)

• Limitations acceptable? If yes, how to point the
  users to these limitations effectively?
• Derivation of RINEX files from real-time streams
  possible for everyone needs some regulation or
  clarification Who is allowed to upload files
  derived from real-time data streams?
• ? Recommendation 2 highrate file creation and
  upload to the GDC in one hand, ideally at the
  broadcasters side