ATLAS Tier 2 Paths Within ESnet

1
ATLAS Tier 2 Paths Within ESnet

• Mike OConnor
• ESnet Network Engineering Group
• Lawrence Berkeley National Lab
• moc_at_es.net

2
ATLAS Tier 2 Connectivity

• Typical ATLAS tier 2 site connectivity should be
  via Abilene and ESnet in order to reach
  Brookhaven Lab.
• ESnets Goal is that DOE Lab ? Univ.
  connectivity should be as good as Lab ? Lab and
  Univ. ? Univ. W. E. Johnston

3
ATLAS Tier 2 Support

• The role of ESnet is to provide networking that
  supports and anticipates the needs of the Office
  of Science Labs and their collaborators.
• ESnet support for ATLAS tier 2 site connectivity
  makes high bandwidth and fault tolerant paths
  between BNL and Abilene an absolute priority.
• Planning is underway to upgrade BNLs ESnet OC48
  circuit to a multi 10G lambda fault tolerant
  Metropolitan Area Network (MAN). (Targeted for
  Q1 2006)
• Today ESnet and Abilene peer in Atlanta, Chicago,
  New York and Sunnyvale with a combined bandwidth
  of 18.2Gbps.These four locations comprise a high
  bandwidth and fault tolerant interconnect between
  these two networks.

4
ESnet Provides Global High-Speed Internet
Connectivity forDOE Facilities and Collaborators
(Spring, 2006 Configuration)
Japan (SINet) Australia (AARNet) Canada
(CAnet4 Taiwan (TANet2) Singaren
CAnet4 France GLORIAD (Russia, China)Korea
(Kreonet2
MREN Netherlands StarTapTaiwan (TANet2, ASCC)
PNWGPoP/PAcificWave
CERN (USLHCnetCERNDOE funded)
SEA
ESnet Science Data Network (SDN) core
ESnet IP core
NYC
AU
MAE-E
SNV
CHI
Equinix
PAIX-PA Equinix, etc.
SNV SDN
ATL
SDSC
AU
ESnet IP core Packet over SONET Optical Ring and
Hubs
ALB
ELP
International (high speed) 10 Gb/s SDN core 10G/s
IP core 2.5 Gb/s IP core MAN rings ( 10
G/s) OC12 ATM (622 Mb/s) OC12 / GigEthernet OC3
(155 Mb/s) 45 Mb/s and less
42 end user sites
international RE peers
Office Of Science Sponsored (22)
commercial and RE peering points
NNSA Sponsored (12)

ESnet core hubs
IP
Joint Sponsored (3)
Other Sponsored (NSF LIGO, NOAA)
high-speed peering pointswith Internet2/Abilene
Laboratory Sponsored (6)
5
ESnet Abilene Peering points
paths do not include all router hops Example
site logos chosen at random.
6
Abilene Routing Toward ESnet
Abilene Interior Gateway Protocol (IGP) mileage
based metrics determine the best exit path toward
the ESnet. This common method also known as hot
potato routing it will always compute an exit
path to the nearest peering point.
7
ESnet Routing Toward Abilene
Abilene Multi Exit Discriminators (MEDs) are used
by ESnet to choose the best exit path toward
Abilene. These MEDs are based on the same IGP
metrics used in the interior of their network.
This creates a symmetric path back to Abilene.
8
32 Avenue Of the Americas NYC
9
Chicago
10
Current Load in Chicago
11
Sunnyvale Atlanta
12
Future Directions

• Long Island MAN Ring
• ESnet Lambda Infrastructure

13
ESnet LI MAN Ring

• 2 ?s (10 G circuits) delivered as two 10 GigEther
  circuits
• 8 X current BNL bandwidth
• Physically diverse fiber paths forming a ring
  topology.
• Will be used as a10 Gb/s production IP and a10
  Gb/s path for SDN circuit services
• Services switch over to the other service path
  during a circuit failure.

?2 SDN/circuit
Circuits based on WDM rings
10 Gb/soptical channels
?1 production IP
SDN Science Data Network
14
ESnet 2010 Lambda Infrastructure and LHC T0-T1
Networking
TRIUMF
CERN-1
CANARIE
Seattle
CERN-2
Boise
CERN-3
BNL
Chicago
Clev
New York
Denver
Sunnyvale
KC
Pitts
GÉANT-1
FNAL
Wash DC
Raleigh
Tulsa
LA
Albuq.
Phoenix
GÉANT-2
San Diego
Atlanta
Dallas
Jacksonville
El Paso - Las Cruces
Pensacola
NLR PoPs
Baton Rouge
Houston
ESnet IP core hubs
San Ant.
ESnet Production IP core (10-20 Gbps) ESnet
Science Data Network core (10G/link)(incremental
upgrades, 2007-2010) Other NLR links CERN/DOE
supplied (10G/link) International IP connections
(10G/link)
ESnet SDN/NLR hubs
Tier 1 Centers
Cross connects with Internet2/Abilene
New hubs
15
Conclusions

• ESnet provides an infrastructure and services
  that are critical to ATLAS science
• ESnet is well positioned to provide essential
  network services to the LHC tier 2 community
• ESnet is working on providing the DOE mission
  science networking requirements with several new
  initiatives and a new architecture