The Evolution of ESA Ground Station Communications to Internet Protocol

1
The Evolution of ESA Ground Station
Communications to Internet Protocol

Manfred Bertelsmeier, Gioacchino Buscemi, Marko
Butkovic, Fakhri Mamedov, Jacobo Matute Issue
July 15, 2002
2
EXCITE Project Drivers

• EXCITE ESTRACK X.25 Comm. to Internet
  Technology Evolution
• ESTRACK / OPSNET strategies
• convergence to single protocol, use of CotS
• IP protocols world-wide de-facto standards
• IP standardised for SLE support
• IP support integral part of CotS building blocks
  for TTC systems
• (vs. X.25 as extra / exception with unknown
  future)
• Control Center internal support already via LAN,
  TCP/IP
• MOC, SOC, SSC, SDC links, real-time Extranet
  support already via routers, TCP/IP
• Current packet switched WAN nearing crossroads to
  complete overhaul
• Future of X.25 parts and support

3
ESTRACK OPSNET Links Before and After Migration
ESA internal LANs
ESA internal LANs
point-to-
point-to-
Sim
Ref. Stn
point
point
Sim LAN
D
Internet
Internet
Ref. Stn
Extranet
LAN
Extranet
Sim LAN
LAN
links
links
ESTRACK Security Perimeter
Firewall
Firewall
Firewall
Firewall
OPSLAN Core
OPSLAN
Router
STC
MCS
Sim
NCTRS
Routers
NCTRS
Routers
MCS
Sim
client
A/B
A/B
A/B
STC
A/B
A/B
A/B/D
client
OCC NetCore LAN
Router A/B
OCC ISS node (X25)
leased lines
leased lines
Current
ISDN
ISDN
Target
prime / backup
prime / backup
Router A/B
Station ISS node (X25)
Station LAN
STC
TMP
TCE
RNG
Router
server
STC
TMP
TCE
RNG
Server
requires change
MC LAN
4
EXCITE Migration Strategy

• Boundary conditions
• no impact on missions currently in orbit or
  before LEOP (e.g. ERS, XMM, Cluster, ENVISAT,
  Integral)
• support to future missions with new system
  (success-oriented Rosetta, LEOP in 2003)
• Concept
• upgrade systems so that they can support current
  and future mode concurrently, subject to dynamic
  reconfigurations
• implement dual support capability
• Context
• maximum alignment with New Norcia Deep Space
  Station implementation, Maspalomas upgrade and
  ESTRACK stations back-end modernisations

5
Principle Scenarios During Migration

6
EXCITE Phases

• Phase 1 - Preparation and Verification
• software adaptations, testbed, end-to-end proof
  of concept, testing in Rosetta scenario, MEX
  scenario (high speed TM), including roll-out in
  stations subject to back-end upgrades
• IMPLEMENTATION REVIEW
• YOU ARE HERE
• Phase 2 Field deployment of dual capability
• completion of control center and stations
  upgrades
• Phase 3 - Mission transitions
• transit actual operations support from X.25 to
  IP, adapted to mission / station use profiles
• Phase 4 - Completion
• withdrawal of OPSNET packet switching equipment

7
Communications Requirements

• Services
• WAN digital voice OCC - Stations (ca. 10...12
  kbit/s)
• WAN data OCC - Stations (up to few hundred
  kbit/s)
• TM, TC, STC client/server, orbital data, GPS,
  auxiliary data, service management, network
  management
• LAN data transit to / from OCC all remaining
  data exchanges inside station, incl. MC, UPS,
  BMS, FM (e.g. NNO)
• Security
• Capabilities
• near non-stop availability -- reliability,
  redundancy, resilience
• capacity -- performance, modularity, scalability
• throughput -- performance, prioritisation,
  congestion management
• Environment
• WAN circuits with delay and errors (benchmark
  400 ms delay one way, BER 10 exp-7 both
  ways)

8
Protocol and System Features
L-5,6,7
L-4
Transport
L-3
Network
L-2
Data Link
L-1
Physical
Sink
Source
Network
9
Architecture and Design

• IP address space 32 contiguous Class C networks
• (starting range 195.74.160.0/19)
• Domain ops.esa.int. DNS server based, localised
  exceptions
• IP WAN as overlay over packet / FR network
• 3 functions access, distribution, core
• access structured cabling, switched Ethernet
  10/100 Mbit/s, VLANs
• distribution, core inter-VLAN routing, ACLs,
  gateway routers (incl. voice codecs)
• Redundancy
• dual backbone / edge LAN switches, hot-swappable
  modules, dual power-supplies, spanning tree,
  Port-Fast, TTC chains distributed over prime /
  alternate LAN switches
• dual routers, using hot standby routing protocol
  HSRP
• Routing
• Highly available and fully dynamic EIGRP based
  routing architecture

10
Architecture and Design - contd

• Technology
• Cisco, low to enterprise range
• previous investments in WAN, LANs, NMS, and in
  support
• VoIP, X.25 over IP, native ISDN capability
• Intelligent routing (EIGRP)
• Quality of Service (QoS)
• Vast range of supported network technologies
• Voice
• final target voice over IP.
• Voice over FR may stay in use during overlay
  network phase.

11
Architecture and Design contd

• NMS Concept
• Determined by Managed Network Elements Service
  Level Scheme
• Key Elements Alarm System, Proactive Performance
  Monitoring, Troubleshooting Capabilities, Service
  Level based Reporting (incl. historical data)

12
Implementation / Optimisation

13
Implemented Features / Performances

• Communications Systems
• Distributed dynamic routing solution
• Hot-Standby Router Protocol (HSRP) for high LAN
  segment availability
• Tuned Frame Relay interface between Cisco routers
  and Netrix nodes
• Robust Quality of Service system for transparent
  coexistence of different traffic types
• Efficient recovery from any single failure
• Fast convergence times
• Provisions for Voice over IP integration
• Subscribers
• Feasible UNIX system configurations under Sun
  Solaris 2.6 and above
• Tuned TCP stacks to cope with high-delay, high
  BER environments
• End-to-End Connections
• Stable performance for real-time telemetry at
  rates up to 256 Kbps with RTT of 800 ms and BER
  of 10exp-7.

14
Representative Case - New Norcia Communications

15
Test Scenarios

• Communications Bearer Testing
• In-house, precisely modelled transmission chain
  OCC to Ground Station
• Proof of concept for connectivity, availability
  and dual protocol use
• End to End Testing
• (with New Norcia Ground Station as part of
  station acceptance)
• Rosetta, Mars Express, individual cases
• TM/TC data at highest mission TM rates
• Both X.25 and TCP/IP as underlying transport
  protocol
• dto., combined cases
• Concurrent transfers of offline and online TM
  over TCP/IP
• Verification of simultaneous transfers of X.25
  and TCP/IP data

16
Verification and Test Results Communications
System(Single Station Case)

• Connectivity checks between station VLANs (NNO)
  ?
• WAN connectivity checks (ESOC NNO)
  ?
• Remote network management (ESOC NNO)
  ?
• Emulation and testing of various failure recovery
  scenarios
• WAN link failures 2-3 s
  recovery ?
• Router failures 7-12 s recovery
  ?
• Router-ISS interface failures 7-12
  s recovery ?
• Router-LAN interface failures 7-12 s
  recovery ?
• LAN segment failures 1-2
  s recovery ?
• LAN switch failures 5-12 s
  recovery ?
• ISS node failures
  ISDN backup ?

17
Verification and Test Results End-to-End, New
Norcia Case

• Online retrieval
• over X.25 maximum mission bit rates achieved
• over TCP/IP maximum mission bit rates achieved
• Offline retrieval
• over X.25 restricted by physical interface bit
  rate (384 Kbps) and TP2 windowing about 320
  Kbps
• over TCP/IP restricted by the physical interface
  bit rate (320 Kbps) - about 290 Kbps
• Concurrent offline / online TCP/IP retrieval
• Quality of Service scheme applied
• Online TM transmitted nominally, offline TM rate
  capped at boundary of the allocated bandwidth
  class
• Concurrent IP / X.25 transfers
• CIR option available in the ISS used to provide
  traffic shaping per virtual circuit
• Non-preemptive coexistence of IP over Frame Relay
  and X.25 traffic

18
Summary and Outlook

• EXCITE background and concept
• Context with existing IP networking, NNO
  implementation and ESTRACK modernisation
• Principles of requirements, architecture, design,
  implementations
• Migration strategy, phases, timeline aspects
• Developments, achievements, results
• Further work
• Continue training for 1st / 2nd line teams for
  Rosetta and MEX LEOP hybrid
  ESTRACK configurations
• Upgrade remaining ESTRACK stations (Perth,
  Villafranca, Kiruna)
• Constant alignment with mission model / stations
  works timelines
• Consolidate VoIP in line with ESOC Intercom
  upgrade