ARNES experience with obtaining and lighting dark fiber

1
ARNES experience with obtaining and lighting dark
fiber

• Avgust JaukArnesavgust.jauk_at_arnes.siCEF
  Networks workshop, Prague, May 15th, 2009

2
Agenda

• Arnes
• Requirements for the network
• Tendering procedure
• Fiber
• WDM equipment
• Experience with WDM and dark fiber
• Plans

3
Arnes

• Arnes Academic and Research Network of Slovenia
• Public institute
• Service provider for research education in SI
• Network
• Services
• Mission to provide efficient infrastructures and
  services in order to allow Slovenia equal
  opportunities for cooperation in the unified
  European research and educational area.

4
Arnes User Community

• Closed user group!
• 1.100 connected institutions
• Universities, institutes, research laboratories
• Museums, schools, (digital) libraries
• 200.000 individual users

5
Arnes Backbone, January 2003
6
Requirements for the new network

• Support for ordinary as well as advanced use
• Advanced users
• Research experiments, Grids,
• Physics, Chemistry, Meteorology,
• Business Continuity, Disaster Recovery
• Data replication, remote backup,
• Hybrid network model
• IP (IPv4, IPv6)
• QoS, multicast
• Dedicated point-to-point links (Gbps, 10Gbps)

7
Requirements for the new network

• Performance
• n GE between smaller PoPs
• n 10GE between main PoPs
• Availability
• Minimize outages fiber routing, equipment
• BER lt 10-12
• Affordability
• High capacity leased lines far to expensive!

8
Tendering for dark fiber

• Tenders
• 2000 fiber in Ljubljana
• 2003, 2008 intercity links
• Market failure
• Late in deregulation
• Lack of competition in intercity fiber
• Telekom Slovenije
• Electricity distribution company
• Highway company
• Result
• High prices
• Limited flexibility
• fiber types
• G.652A only water peak!
• But managed to get PMD lt 0.2 ps/sqrt(km)
• Maintenance options
• Problems with repair time in case of aerial
  (OPWG) cables

Redundancy is a must!
9
Leased fiber, May 2009 1200 km
9
10
Requirements for the WDM network

• Specifics Slovenia is a small country, but
• A lot of PoPs (in 24 cities)
• Many PoPs are relatively small, without advanced
  users
• Distance between PoPs up to 100km
• Expensive fiber lease!
• Financial constraints
• Solution
• DWDM only between major PoPs
• CWDM between all PoPs
• Operation over a single fiber strand!

11
Bidirectional use of optical fiber

• One fiber strand used for DWDM between main PoPs
• Another fiber strand to connect smaller PoPs with
  GigE using passive CWDM

DWDM Equip.
DWDM equip.
10 GigE implemented with DWDM
IP router
IP router
Gigabit Ethernet Implemented with CWDM
IP router
Maribor
Ljubljana
fiber pair
Celje
11
12
Requirements for the CWDM network 03

• Performance
• At least 4 channels, 1Gbps
• Price/performance
• Single fiber strand operation!
• Passive device filters only
• CWDM pluggables in routers
• Compact size
• Affordable price

13
CWDM network topology (Sunrise POC)
13
14
Requirements for the DWDM network 07

• Performance
• Future proof (high traffic growth rate!)
• At least 16 channels, 10Gbps
• Availability
• No protection on optical layer
• On L2 and L3 only!
• BER lt 10-12

15
Requirements for the DWDM network 07

• Price/performance
• Single fiber strand operation!
• Support for 10GE LAN PHY, without regeneration!
• OTN was not required
• Support for alien lambdas (Cisco DWDM Xenpak)
• Compact size, green (low power consumption)
• Easy to install, operate and maintain
• Flexible, support for different applications
  (cards)
• Avoiding cheap initial config / expensive
  add-on sindrom

16
DWDM Network Solution

• Single Fiber Working ADVA FSP 3000 with enhanced
  EDFAs

17
DWDM network topology (ADVA FSP 3000)
17
18
Experience with WDM / leased fiber

• General remarks
• A lot of work with tendering, but it pays off
• Dealing with fiber providers can be challenging
• Need for basic knowledge on optical and WDM
• DWDM easier to install and run than expected
• Routers are more difficult!
• Modern metro DWDM solutions can be quite compact
  (7HU, shallow back to back)
• Alien lambda support really works (tested with
  DWDM Xenpaks)
• Problems with pluggable modules for routers
• Form factors (SFP, GBIC, Xenpak, X2, XFP, SFP)
• Vendor locking

19
Experience with WDM / leased fiber

• Operational WDM
• WDM equipment is reliable. BER 0
• Firmware upgrades even for DWDM -)
• History of Tx Rx power very usefull
• SNMP support on DWDM equipment
• SNMP and SFF-8472 (Diagnostic Monitoring
  Interface) support on routers

20
Experience with WDM / leased fiber

• Operational fiber
• Measuring PMD makes sense
• Fiber cuts (constructions, rats), bends,
• Quite a lot of maintenance work
• Aerial fiber
• No problems with stability due to temperature and
  wind
• But problems with shooters and thieves
• Repair can take days
• CWDM DWDM in parallel, Cacti -gt easier
  troubleshooting
• Excellent price/performance

21
Finally enough bandwidth!
22
Backbone alternative view
23
Plans

• Connect cities passed by the fiber
• Add more Bandwidth when needed
• Additional C/DWDM channels
• Upgrading CWDM to 10G speeds and/or DWDM
• Concurrent use of CWDM and DWDM on the same fiber
• 40G/100G
• Looking for simple technology for Point to Point
  links
• CBF - options
• Austria (Maribor/Graz)
• Croatia (Krko/Zagreb)
• Italy (Seana/Trieste, Nova Gorica/Gorizia)

24
Thank You!
24