Dan Magorian

1
Whats MAX Been Up to Lately?Presentation to MAX
membership
Spring 06 Member Meeting

• Dan Magorian
• Director of Engineering Operations

2
MAX Production Net and Research Activities
MAX Activities
Production Net Activities
Research Activities
Internet2 HOPI Test Bed support
DRAGON Experimental Networking
Layer 3 IP Network Services
Layer 1 Optical Services
GIG-EF Research Technologies
NGIX/East, AtlanticWave peer points
Disaster Recovery Colocation
Additional Funded Research
3
Where we were a year ago
4
Where we are Spring 06
State of MD pop, Baltimore
BALT
M40E
1. Original Zhone dwdm over Qwest fiber
2. Movaz dwdm over State Md fiber
3. HHMI dwdm over Abovenet fiber
Ring 4
4. (July) USM dwdm over State Md fiber
Level3 pop, Mclean VA
CLPK
NGIX
Ring 2
CLPK
MCLN
T640
T640
UMD pop, College Park MD
CLPK
MCLN
Ring 1
Ring 3
DCGW
DCNE
ASHB
ARLG
Equinix pop, Ashburn VA
GWU Qwest DC pops, ISI/E Arlington VA pop
5
Has been/will be completed by July 06

• Protected ring out of fiber CLPK Level3 pop
  Mclean
• Built out to Equinix public peering facility with
  help of HHMI (thanks!).
• Redundancy/doubled capacity Qwest ISP via
  Equinix. Pioneered low-cost backup 50mb rate
  vs 200mb with Qwest for Quilt folks.
• After extended legal process, 400M Qwest ISP and
  I2 sold to HHS via Qwest Federal Systems with MAX
  as subcontractor.
• At Equinix, likely will transit IPv6 carrier
  peerings to Internet2, making MAX primary east
  coast IPv6 exchange for them.
• Brought up Cogent ISP through GWU. Anyone
  interested in 25/mb?
• Good uptake on 2.5 10G lambdas to NLR 4
  leased, 6 in works
• Phased out WhatsUp, deployed new Nagios-based
  open-source monitoring and alerting system for
  bgp routes as well as link state.
• Hosted I2 performance workshop, built portable
  10G test pcs available to members, deployed new
  noc and backup systems.
• Hired 2 engineers, search for AtlanticWave
  engineer in progress.

6
Projects Underway FY06-07

• 10G L3 backbone upgrade (more detail on following
  slide)
• Baltimore dwdm ring. USM selected MRV, was ready
  to go, but MDOT wanted in on project at 11th
  hour with additional sites and lambdas. Project
  expected to be online July. There will also be
  new 10G-capable Force10 switches at ring
  locations.
• More use of external lambdas, dropping costs, new
  vendor Ekinops (French) has cost-effective dwdm
  add-in gear.
• Atlantic Wave switch partnership with Ciena, will
  be used for upcoming GEANT peering (more detail
  on following slide)
• Advanced Exfo chromatic dispersion/polarization
  mode dispersion test gear just ordered, needed as
  we provision 10G waves over contributed unknown
  fiber. Will get additional test gear in 07 for
  sonet ethernet analyzers. Loanable members.
• Continuing measurement and alerting
  infrastructure buildout for redundancy, alerting
  on optical, full info tracking database.

7
10G L3 backbone/Abilene backup upgrade

• Sun still intending to join MAX at 10G, but their
  grid computing project is delayed. Still expect
  to see 10G connectors in FY07.
• However, Abilene backup with another gigapop,
  either MAGPI or SOX, may happen sooner in 07.
  Will need 10G for this, also NLR wave or other
  connection.
• Need to move beyond current gige Abilene
  redundancy with CLPK and LVL3 routers as grow, to
  avoid having eggs in same basket.
• Hard part has been getting Juniper 10G T640
  router interfaces at reasonable cost. Finally
  found one pair used.
• Finally convinced Juniper that need to offer 10G
  ints at high discount to all of higher ed to
  avoid missing out on 10G market. They will offer
  this to Quilt members at about 70 discount in
  1-2 months..
• The way lambda costs have been dropping, only
  makes sense to provision 10G lambdas these days,
  and do 2.5G as subrate. FY05 10G transponder
  pair 77k. FY06 54k. Now, just paid 24k.
  Now as soon as we can get Juniper to follow suit

8
Mid-Atlantic Crossroads
National/International Peering Networks
Regional Network Participants
NGIX-East
GIG-EF
Univ System Md
Network Virginia
Institutional Connectors
9
Strong Carrier Partnerships

• With 7 MAX pop locations around region,
  participants can connect at one or more locations
  via dark fiber or lit services via aggressive
  low-cost carriers, as well as traditional telcos.
  
• 80 growing of participant circuits ethernet.
• Strong partnerships with carriers (12 growing!)
• Qwest, Level 3, Fibergate, Starpower/RCN, Yipes,
  Allied Telecom, AboveNet, Looking Glass Nets,
  Verizon, MCI, ATT, Comcast.

10
MAX and NGIX/E upstream network peerings

• Abilene (diverse 2.5 gb primary /1 gb backup, 10
  gb)
• vBNS network (1 gb)
• Defense Research Exp Net (DREN) (1 gb)
• NASAs NISN and NREN (1 gb ea)
• USGS DOI net (1gb)
• Qwest DIA ISP service offering (1 gb, 2nd 1 gb)
• Cogent ISP service (1 gb)
• National Lambda Rail, potentially NLR L3 service.
• GEANT2, European Abilene, (10 gb, May)
• Energy Sciences net (ESnet) (1 gb, becoming 10
  gb)
• Atlantic Wave distributed peer fabric, (summer 06)

11
Atlantic Wave over NLR lambda
CHI
LON
NYC
STK
SEA
WDC
TOK
CER
LAX
ATL
AMS
A-Wave
MIA

• East Coast confederation of
• Network Exchange points
• Production level IP peering
• Experimental Light Path services

AUS
Sao Paulo (SPB)
12
Atlantic Wave switch partnership with Ciena

• Awave switch was in FY05, carried over to FY06
  budget. Best guess, real costs were unknown.
• SURA funding of Awave 10G lambdas over NLR is
  protracted process due to contractual issues with
  4 universities.
• Meanwhile, GEANT 10G sonet circuit arriving in
  May needing to be terminated using GFP framing.
  We need to move forward.
• We held sonet switch vendor presentations by
  Ciena, Nortel, Sycamore, and Alcatel (used by
  GEANT), got quotes.
• Ciena out of blue decided to gtgtdonateltlt
  CoreDirector switch sonet interfaces to MAX, do
  co-development gmpls MOU with DRAGON. Were going
  donate 5 to Internet2, now MAX instead.
• Talked Ciena into 2nd free switch, more
  interfaces 1 production, 1 research. Will need
  to pay FY07 only for 6 newly released 10G
  ethernet ints, maint, training, etc.
• Very signification donation to MAX, value 4-500k
  for 12 OC192 ints, 16 OC48 ints, 20 gige ints in
  1 full height 1 half chassis.

13
MAX expanded customer colo and disaster recovery
sites

• Disaster recovery sites have been a continuing
  request from members at MAX meetings. People
  want affordable colo space to host their own gear
  offsite, yet accessible.
• In response, we have investigated 4 options
  trade deals with Northrup Grumman and
  InterAmerican Development Bank, both new members
  with colo facilities visited GWUs VA facility,
  and striking a much better price per sq ft and
  per amp for a lot of space on the second floor
  of Level3 pop in Mclean. All still in progress,
  Level3s would be most convenient if they can
  bring price down.
• Meanwhile, Max unexpectedly acquired a fully
  equipped 1200 sq ft machine room across the hall
  with generator, dual large UPS, overhead power
  distribution, underfloor dual AC units. Will
  hold 38 cabinets. Need because MAX test lab out
  of space, but would like to make available to
  members also. Quote out to ring out 8400 fiber.
• Would charge folks upfront for standardized
  cabinets, then lease lambdas back to their
  institution so have private dedicated bandwidth
  and looks like part of their network with no MAX
  L3 in between.

14
(No Transcript)
15
1200 sq ft, can hold 38 cabinets, independent
access to outside ramp
16
One of two large Liebert AC units feeding
underfloor cooling
17
Dual 50 kva UPS with generator feeding overhead
dual 208 VAC power rails
18
(No Transcript)
19
(No Transcript)
20
Thats fine for folks who can use DR in the DC
area, but what about farther out?

• Have begun talks with UMBC about a separate
  machine room space to be leased to MAX that could
  hold around 28 cabinets.
• Location (40 miles from College Park) is good,
  already on new Baltimore dwdm ring so can
  provision colo lambdas.
• UMBC was original UMD partner in BWI gigapop in
  vBNS days (97), so long history of good working
  relationship.
• Power is good, space is good, but they need to
  install additional cooling for new space. Study
  underway, hopefully reasonable cost
• Intend to have standardized offering both machine
  rooms for about 1500/cabinet one-time plus
  500/cabinet/ month. Wanted to be free as
  benefit of membership, but needed recoup lease,
  maint, etc
• Will likely lease private lambdas, mpls tunnels
  over shared 10G.
• If can have online summer (8400) and early fall
  (UMBC), hoping to have good uptake in cabinet
  leases. Are you interested?

21
MAX Video Lab/Access Grid node

• DRAGON has successful ongoing work with ISI/E and
  others for development of UltraGrid video codecs
  for HD over IP.
• This work now is crammed into MAX bank vault
  conf room.
• Also, we have lot of requests from MAX membership
  (eg, Holo- caust Museum this week) for help with
  video projects like multicasting events. Right
  now, nowhere to test against.
• Idea is to take old MAX lab after equipment moved
  to new machine room, convert it to video lab and
  access grid node. Already well equipped with
  separate ac, good power, carpet and good
  lighting. Hope to have new video lab in by fall.

Ladan Gharai of ISI and Chris Tracy of MAX
compare parking permits over high def (better
than life)
22
Recruiting for Engineering and Operations
Advisory Group

• Ongoing MAX Strategy and Research Advisory
  Groups.
• Have had Core Engineering Group for years, but
  that needs to be expanded and meet more regularly
  (quarterly).
• Want to offer chance for techies to help us plan
  future MAX engineering projects and directions.
• Caution this is not for management folks and
  those wearing suits. It will be highly
  technical. We hope that the lead network
  engineers of each institution will be interested.
• The intent is for some sessions to have
  presentations of plans, others to have
  brainstorming sessions on selected topics, and
  periodically to have invited vendor
  presentations.
• Interested? Please mailto//magorian_at_maxgigapop.n
  et.

23
Thanks!