Beyond the Line

1
Beyond the Line

• Swedish Operators Forum
• Stockholm
• November 26th 2002

2
Per Gregers Bilse

• bilse_at_networksignature.com
• Systems software and compiler development for
  Borland (Turbo Prolog), ports to OS/2 and UNIX.
• Network Engineer / Architect / Manager /
  Director, EUnet, Amsterdam, The Netherlands.
• Technical Leader, Cisco Systems, London, UK
• Backbone Director, Metromedia / AboveNet, London,
  UK
• Consultant and Contractor, London, UK

3
NETWORK MONITORING WELCOME TO THE STONE AGE

• severe lack of monitoring and management tools at
  Level 3
• existing tools work in terms of "lines" and
  "interfaces these are level 2 entities
• more often than not, "network management" is
  "server management
• focus on server load, uptime, packet loss,
  latency, services
• all have Editors Choice award from PC Magazine
  or similar
• other efforts experimental, conceptual, off
  target, dont scale, and/or very expensive

4
STILL STONE AGE

• is this a people problem?
• software gurus understand data structures, bytes,
  port and protocol numbers they take the OSI
  reference model literally (and even try to
  implement it)
• network gurus understand architecture, routing,
  systems, queueing, congestion they see the OSI
  reference model as a conceptual framework (and
  die-hards swear by the DARPA four-layer model)
• the two sides rarely talk

5
Something made by a software guru ...
6
The world will now come to an end.

7
Something made by a network guru
8
STILL STONE AGE

• network guru tools look at number of
  announcements, address space covered, routing
  stability, etc
• important, but its limited what one can do with
  the information
• software guru tools focus on Level 4, inside the
  packet, bypassing Level 3
• Level 4 not interesting, the data has to be
  carried anyway
• give or take a little, the best they do on Level
  3 is to collect lists of IP addresses
• SNMP offers MIBs for everything, but this is not
  useful
• performance issues makes SNMP useless for large
  volume data
• even if performance was OK, nobody can do
  anything useful with the data

9
THE STONE AGE IN CONCLUSION

• network abstraction is poorly understood outside
  the core networking community
• Level 2 is all about MAC addresses
• Level 3 is all about IP addresses
• Level 4 is all about protocol and port numbers
• Etc
• software developers dont embrace lateral
  abstractions such as the Autonomous System
  because it doesnt exist in the OSI model
• there is a perceived problem of being unable to
  handle large volumes of data
• there is no understanding of the need for real
  time or near real time tools

10
INTRODUCTION TO THE BRONZE AGE

• Network Signature BENTO
• BGP
• Enabled
• Network
• Traffic
• Organizer

11
BRONZE AGE BENTO WHAT IS IT?

• a set of extreme performance server applications
• receives netflow or packet header information
• looks up corresponding BGP attribute information
• aggregates flow information around BGP
  information
• stores aggregated information on disk
• produces graphs and plots from aggregated
  information
• can use any BGP attributes, currently focus on
  paths
• works in almost real time (worst case two minutes
  behind)

12
The innards From raw materials to finished
product
13
Performance

• Prototype developed on low end Linux PC
• 800MHz AMD Duron on VIA686 (PC Chips)
  motherboard
• 256Mb PC100 memory
• Soft RAID on UDMA33 disks
• Many performance evaluations, typical scenario
• three full BGP feeds
• mix of real and simulated netflow information
  equal to 1.8Gbps source traffic
• 10-20k active paths on ring
• CPU load is variable
• can in any case handle data for several Gbps of
  unsampled traffic on fast PC
• trivial to bolster with retrospective sampling
• PNG image compression takes considerably more CPU
  than most other things.

14
Data extractions

• we store, and work with, the hardest part full
  AS path
• peer and/or home AS is easy
• we can extract anything we like from the path,
  including
• peer AS
• home AS
• in fact, any AS as home or transit
• actually, any set of ASs as peer, home,
  intermediate, and/or transit
• we can sort and group on
• path length
• packet count
• traffic volume
• protocol group, eg paths with a lot of ICMP

15
AS spectrum (excerpt)
16
Complete peer spectrum with summarised traffic
17
Complete paths, sorted
18
Top 20 busiest paths, difference between now and
5 minutes ago

19
One hour history
20
Possible uses

• network planning and optimisation (next slide)
• real time network monitoring, detection of
  anomalous/malicious traffic (DOS)
• can do a lot with fancy colours
• future extensions with rule-based traffic
  evaluation
• exchange case what if I were to peer privately?
  Connect to another exchange?
• the impossible dream A Network Signature.
• we have both routing information and
  corresponding traffic information
• compare to historical data
• five minutes ago
• one hour ago
• one week ago
• one month ago
• even this time last year
• result are we normal today?

21
The big question
22
Availability

• currently running in test on AMS-IX with two
  pilot customers exchange-based service is free
  for (at least) one year for all members
• supports cisco and Juniper netflow version 1 and
  5 other formats (eg sFlow) trivial to implement
• corporate/private multirouter version to be
  arranged
• currently licensed as a supported service, to
  avoid cost of manuals, technical support,
  multiple OS version support, complicated
  contracts, etc DUE TO CHANGE
• open to suggestions, ideas, cooperation, etc
• native, real time application TBA

23
How to use

• register router(s) with the BENTO software (web
  interface)
• IP address
• optional list of SNMP interface numbers
• AS number
• sample rate
• set up BGP session(s) with BENTO-BGP daemon
• configure netflow export set IP address,
  version, and cache timeout
• configure netflow accounting on relevant
  interfaces
• sit back, relax, enjoy

24
General Cisco configuration

• interface fe0/0/0ip route-cache flow
• ip flow-export version 5
• ip flow-export destination 193.148.15.2 12345
• ip flow-cache timeout active 1

25
General Juniper configuration, 1 of 3

• interfaces
• fe-0/0/0
• unit 0
• family inet
• filter
• input SampleAll

26
General Juniper configuration, 2 of 3

• firewall
• filter SampleAll
• term all
• then
• sample
• accept

27
General Juniper configuration, 3 of 3

• forwarding-options
• sampling
• input
• family inet
• rate 100
• output
• cflowd 193.148.15.2
• port 12345
• version 5

28
Thanks!

• Special thanks go to
• Job Witteman and the AMS-IX crew
• Alex Bik and Business Internet Trends, bit.nl
• Linux and the cheap PC
• All the people who said it couldnt be done
• bilse_at_networksignature .com