IPv6: DoD Pilot Implementation on DREN

1
IPv6 DoD Pilot Implementation on DREN

• Joint Techs Workshop
• July 2004
• Columbus, OH

Ron Broersma DREN Chief Engineer High Performance
Computing Modernization Program ron_at_spawar.navy.mi
l
2
Context for this briefing

• Historical
• June 2003 DoD CIO issues IPv6 transition
  memorandum
• Target completion 2008
• July 2003 DREN chosen as the DoD IPv6 pilot
  implementation
• Plans to implement in 2004
• Within DoD
• Each of the services (Army, Navy, Air Force)
  developing their own transition plans for the
  operational networks.
• Most will not begin implementation for a year or
  more
• Most will not be complete until after 2008
• DREN is DoDs research network, and is
  transitioning now.
• Chartered to support the DoD HPC community, and
  other RD organizations.

3
DREN Today

• 10 core nodes on OC-48 backbone (CONUS), with
  extensions to Hawaii and Alaska.
• Now updating to OC-192 (10 Gigabit)
• About 100 sites (Service Delivery Points),
  connected at DS-3 to OC-48 rates.
• IPv4 unicast and multicast, IPv6 unicast, and ATM
  services now.
• Dual IPv6 networks (testbed, and production)
• jumbo-clean (i.e. 9K MTU everywhere)
• Multiple security levels.
• Both unclassified and classified networks

4
DREN Map
5
DREN IPv6 History

• 1995-2000
• Ad-hoc tunnels, playing on 6bone.
• Presentation at conferences
• IPSEC (NRL)
• Early implementations (NRL stack)
• Jan 2001 -
• DRENv6 testbed
• Native IPv6 (no tunnels)
• Logically separate from DREN IPv4 backbone
• OC-3 interconnects (ATM PVC mesh)
• 8 core nodes (Cisco routers dedicated to IPv6)
• Sites connect via PVCs (native IPv6), or tunnels.
• Peering with IPv6 enabled ISPs
• DREN sites encouraged to connect and participate
  in testing and experimentation. Many tests
  conducted, many lessons learned.
• If you build it, they will come
• 2002
• New DREN2 backbone contract (MCI) includes IPv6
• Jul 2003
• Selected as DoD IPv6 pilot (details below)

6
DRENv6 testbedLogical Topology
Cisco
AIX-v6
CW
Global Crossing
6TAP
Abilene
Abilene
FIX-West
Hurricane Electric
LAVAnet
TIC
WPAFB
Dayton
ARL
NTTCom Verio
JITC
HP
Aberdeen
Tunnel broker
San Diego
WCISD
AOL
SD-NAP SDSC
SSC San Diego
Wash D.C.
SPRINT
HICv6 (Hawaii)
NRL
Vicksburg
Albuquerque
SSC Charleston
SSAPAC
ERDC
AFRL Kirtland AFB
Stennis
vBNS
ATM PVC (OC-3)
NAVO
IXP
Core Router
tunnel
site
ISP or BGP Neighbor
7
Lessons from Testbed experience(state of things
1 year ago)

• Our customer sites find little or no incentive to
  run IPv6 (LAN administrator perspective).
• There is no capability or feature of the Internet
  that you can't do today by not running IPv6. 
• Turning it on brings additional complexity, and
  has a learning curve.
• Users arent asking for IPv6.
• There is no immediate "win" to transitioning to
  the new protocol.  The payoff is long-term. 
  External incentives will be needed to encourage
  near term adoption and transition.
• If you build it, they wont necessarily come
• Many commercial security components (like
  Intrusion Detection Systems, Firewalls, Security
  Scanners, etc.) don't yet support IPv6, so it is
  very difficult to deploy the technology to our
  sensitive DoD networks in a secure fashion.

8
DREN as DoD IPv6 Pilot

• DREN is in a unique position to serve as a DoD
  IPv6 pilot
• Experience running IPv6 WAN.
• RD environment familiar with technology
  insertion, and being a pioneer.
• New contract includes IPv6 support in the WAN (we
  just have to turn it on).
• Management support.
• Have the means to deal with the challenges.

9
FY04 DREN IPv6 Initiative

• DoD IPv6 Pilot network
• Goals for 2004
• IPv6 enabled DREN infrastructure (all Service
  Delivery Points, the Wide Area Network, the NOC).
• Facilitate IPv6 deployment into infrastructure at
  HPC user sites and DREN user sites.
• IPv6 enabled HPCMPO, HPCMP funded assets and
  services, HPCMP user community support
  applications, selected user application
  candidates.
• Performance and Security as good as existing IPv4
  service.
• Provide product feedback, lessons learned,
  published via web.
• Functional Areas in this project
• IP transport and infrastructure Ron Broersma,
  Navy
• Infrastructure services Phil Dykstra, WCI
• Network Management Tom Kile, Army
• Security Doug Butler, OSD
• Applications Ralph McEldowney, Air Force
• Planning for the Future Ron Broersma, Navy
• HPC Community Involvement John Baird, OSD

10
Transition Strategy (Notional)

• Start with core, and work out to the edge
• Hybrid (Dual Stack) infrastructure
• Minimize need for tunnels, translators, and other
  transition schemes

S
A
S
Site LAN
A
Site LAN
S
Site LAN
A
S
A
S
WAN (DREN)
Site LAN
NOC
Application
Internet
A
S
Server
11
Goal 1 IPv6 enabled DREN infrastructure (all
Service Delivery Points, the Wide Area Network,
the NOC).
Complete

• All 100 WAN routers (Juniper) upgraded to JunOS
  6.1 to support IPv6.
• Includes all Service Delivery Points (SDPs) and
  DREN Core Nodes (DCNs).
• Connectivity to Internet (IPv6) via DREN Testbed.
• Backbone is now IPv6 enabled and ready to bring
  production sites online.
• Sites already turned up HPCMO, SSC San Diego,
  ARL, NRL, ERDC, Indian Head, Quantico, Norfolk,
  Charleston, DREN NOC.
• Tunnel Brokers (Hexago) for each network.
• Testbed, DREN, S/DREN
• Network and Users conferences are IPv6 enabled.
• Cleanup readdressed entire WAN to conform to
  new addressing plan.

12
Goal 2 Facilitate IPv6 deployment into
infrastructure at HPC user sites and DREN user
sites.
Complete (at HPC sites)

• Road show to 13 sites (to date)
• ARL, ASC, ERDC, NAVO, AHPCRC, ARSC, MHPCC, SMDC,
  NRL-DC, RTTC, HPCMPO, DREN NOC, HPC CERT.
• Briefing for Executives, Management, and
  technical staff.
• Get buy-in from all levels of management.
• Incentivise sites to upgrade local infrastructure
  and systems.
• Offer assistance, resources, training.
• Establish transition team within each
  organization.
• ASC went live on 26 June. ARL in August.
  Others to follow.

13
HPC sites being IPv6 enabled
14
New Challenge

• Before
• Little incentive to transition to IPv6
• Now
• No real resistance.
• Site visits are paying off.
• New Problem
• Transition to IPv6 is just one of many new
  priorities (security, new systems, etc).
• Efforts with near term return on investment (ROI)
  get priority. IPv6 transition has far term ROI.

15
Goal 3 IPv6 enabled HPCMPO, HPCMP funded
assets and services, HPCMP user community support
applications, selected user application
candidates.
Continuing Effort

• HPC Program office
• done
• HPC assets/services
• first ones starting to go live now
• HPC support applications
• Kerberos mostly complete
• IDS done
• Web sites (InfoEnv, OKC) Fall 04
• User applications (mostly 3rd party)
• Discovery process well along
• Actual transition depends on vendor/developer
• Recent breakthrough FlexLM (Macrovision)
  committed to IPv6 support

16
Goal 4 Performance and Security as good as
existing IPv4 service
Success

• Performance
• IPv6 performance within 0.3 of IPv4 on various
  stress tests.
• Security
• Through workarounds, we can achieve equivalent
  security posture.
• Catching attacks, blocking viruses.
• DSAWG Review no issues.

17
Performance Results

• Phil Dykstra (on DREN2 pilot net)
• Using iperf, SSC San Diego, CA to ARL
  Aberdeen, Maryland, MTU 9k, I get about 567
  Mbps with IPv4, 565 Mbps with IPv6. So at first
  glance, performance seems nearly identical (minus
  the extra header overhead of course).
• Done between 2 Linux machines on opposite coasts
  connected to DREN OC-12 sites.
• 10Gb-E testing at HPC Center, sending a 4 Gb/s
  stream from Linux with 10Gb-E NIC.
• 3939.8044 Mbps UDP single stream (IPv4)
• 3930.6234 Mbps UDP single stream (IPv6)

18
DoD Security Model

• Defense in Depth
• Protections at multiple levels
• Problem How to securely deploy IPv6 in DoD
  without these components.

S
Scanners
LAN
Firewall
IDS
ACL
WAN
ACL
IDS
Internet
19
Lack of Security Features (Examples)

• Router Access Control Lists (ACLs)
• Juniper doesnt support tcp established
• Vulnerability Assessment (Scanners)
• ISS doesnt support IPv6 and has no published
  plans to do so.
• NESSUS doesnt support IPv6 (yet)
• Intrusion Detection Systems
• If we want IPv6 support, we have to add it
  ourselves.
• Juniper port mirroring doesnt support IPv6
• IPSEC
• Missing in most IPv6 implementations
• Juniper ASPIC doesnt support IPv6 (until much
  later)
• Firewalls
• Until recently, no production quality IPv6
  support
• Netscreen (Juniper)
• no OSPFv3, only RIP
• IPv6 support only available in certain products
• High end products wont have IPv6 support until
  next year.

It is crucial that IPv6 products have equivalent
functionality to the IPv4 world
20
Overcoming the security issue (workaround)

• Use DRENv6 testbed for transit to Internet
• use to peer with rest of IPv6 enable Internet and
  other testbeds
• continue to operate as an untrusted IPv6
  network
• Enable IPv6 on new DREN2 (MCI) production
  network.
• Dual stack everywhere.
• Establish trusted gateways between v6 enabled
  DREN2 and the DRENv6 testbed
• Upgrade HPC Network Intrusion Detection Systems
  (NIDS) to be v6-compliant, monitored by the HPC
  Computer Emergency Response Team (CERT), and
  install at the trusted gateways.
• Install v6 version of standard DREN v4 Access
  Control Lists (ACLs) to protect pilot network to
  same level as IPv4 production network.
• DREN customers receive safe native IPv6 service
  via existing service delivery point (SDP), in
  parallel with IPv4 service.

21
DREN IPv6 transition architecture FY04
To 6bone, Abilene, and other IPv6 enabled ISPs
IPv6 demonstrations (Moonv6)
links run native IPv6 where possible, otherwise
tunnelled in IPv4
DRENv6 (Testbed)
ARL-APG
Native IPv6 backbone
SSCSD
ERDC
Testbed at DREN site
Testbed at DREN site
NIDSv6
NIDSv6
v6 ACL
NIDSv6
v6 ACL
v6 ACL
sdp.erdc
DREN2 (Production / Pilot)
sdp.arlapg
sdp.sandiego
Dual stack IPv4 and IPv6 wide area infrastructure
sdp
sdp
sdp
Goal As secure as the IPv4 backbone
Type A (IP) production service to DREN
sites IPv4 and IPv6 provided over the same
interface
22
Site Security Solution(Example SPAWAR)

• SPAWAR Intrusion Detection System (IDS) modified
  to support IPv6
• Netscreen Firewall operating beta release with
  IPv6 support in parallel with production firewall.

DREN2 (Pilot)
WAN
IPv4 unicast and multicast services IPv6 unicast
SPAWAR Border router (Juniper M20)
IDS
IPv4
IPv6
Netscreen 500 Firewall
Netscreen 208 Firewall
Note Netscreen (Juniper) now has mainstream
IPv6 support for some models.
Production Firewall
IPv6 Firewall (beta code)
switch
to LAN
23
Ongoing Security Effort

• Snort 2.0.1
• Upgraded to IPv6 Ken Renard
• In production use today by HPC CERT
• Snort 2.1.1
• Upgraded to IPv6 and available.
• Unable to get support included in main snort
  distribution.
• IPSEC interoperability testing in Moonv6 phase
  II.
• ACL and Firewall testing in next phase of Moonv6
• LIBNIDS
• Work underway to modify for IPv6. Available late
  summer.
• Kerberos v1.3 (MIT)
• IPv6 updates for DREN release by Ken Hornstein
  (NRL)
• Working on IPv6 for
• DoD CAC with OpenSSL, PKI, OCSP, LDAP

24
Goal 5 Provide product feedback, lessons
learned, published via web
Complete

• DREN IPv6 knowledge base
• https//kb.v6.dren.net
• Open to all DoD (with PKI certificate)
• Online and ready for articles
• Initial articles published
• Challenge getting people to input their lessons
  learned.

25
Large projects with interest in IPv6, using DREN

• Global Information Grid (GIG) related experiments
  (NRL, SPAWAR)
• Future Combat System (FCS) (Army)
• Existing DREN sites, plus 8 new Boeing sites
• E10A Constellation (Air Force).
• Fleet global unified routing architecture (Navy),
  FORCENET
• Military Service Academies
• Train future leaders to expect benefits of IPv6

26
Mobility Utilization

• Transition to support future mobile soldiers
  Force XXI Land Warriors

Helmet mounted computer and display systems,
weapons with video imaging tied to GPS, backpacks
with satellite and ground communication links,
radios, 15 pounds of batteries, and more
computers, all networked with other warriors and
nearby tanks, helicopters, and personnel carriers
27
Mobility Utilization

• Transition to support future mobile Service
  platforms the Command and Control Constellation
  E-10A aircraft

A fully connected array of platform-, space-, and
land-based sensors that use common standards and
communication protocols to relay information
automatically via machine-to-machine interfaces
28
Mobility Utilization

• Transition to support future mobile sensor webs
  blue-water and littoral sensor webs for FORCEnet

29
Backup
30
DREN performance measurement tools

• DREN AMP
• Active Performance Measurement system
• IPv6 updates Phil Dykstra
• nuttcp 4.0 (NRL)
• TCP performance tester (client/server)
• IPv6 updates Rob Scott (NRL)
• ftp//ftp.lcp.nrl.navy.mil/pub/nuttcp

31
Addressing

• 2001480/32
• /44 reserved for each SDP
• Sites get a /48
• All subnets are /64
• No tiny subnets for point-to-points