Mapping the Internet and Intranets

1
Mapping the Internet and Intranets

• Steve Branigan, Hal Burch, Bill Cheswick
• ches_at_lumeta.com

2
Motivations

• Intranets are out of control
• Always have been
• Highlands day after scenario
• Panix DOS attacks
• a way to trace anonymous packets back!

• Internet tomography
• Curiosity about size and growth of the Internet
• Same tools are useful for understanding any large
  network, including intranets

3
The Original Project

• Long term reliable collection of Internet and
  Lucent connectivity information
• without annoying too many people
• Attempt some simple visualizations of the data

• movie of Internet growth!
• Develop tools to probe intranets
• Extended database for researchers

4
Uses for the Internet data

• topography studies
• long-term routing studies
• publicly available database
• (open source) for spooks
• interesting database for graph theorists
• combine with other mappers to make an actual map
  of the Internet

5
History of the Project

• Started in August 1998 at Bell Labs
• April-June 1999 Yugoslavia mapping
• July 2000 first customer intranet scanned
• Sept. 2000 spun off Lumeta from Lucent/Bell Labs
• June 2002 B round funding completed

6
Related Work

• See Martin Dodges cyber geography page
• MIDS - John Quarterman
• CAIDA - kc claffy
• Mercator
• Enter internet map in your search engine

7
(No Transcript)
8
(No Transcript)
9
Methods - data collection

• Single reliable host connected at the company
  perimeter
• Daily full scan of Lucent
• Daily partial scan of Internet, monthly full scan
• One line of text per network scanned
• Unix tools

10
Methods - network scanning

• Obtain master network list
• network lists from Merit, RIPE, APNIC, etc.
• BGP data or routing data from customers
• hand-assembled list of Yugoslavia/Bosnia
• Run a traceroute-style scan towards each network
• Stop on error, completion, no data
• Keep the natives happy

11
Daily database

• 100-200MB of text
• compresses to 5-10MB
• daily Internet results available from mapping web
  page
• have not checked to see who gets it!
• Saved to different partition, and offloaded to
  other secure computer

12
Traceroute

• Probes toward each target network with increasing
  TTL
• Probes are ICMP, UDP, TCP to port 80, 25, 139,
  etc.
• Some people block UDP, others ICMP

13
Traceroute
Hop 3
Hop 1
Hop 2
Hop 4
Hop 3
14
Send a packet with a TTL of 1
Hop 3
Hop 1
Hop 2
Hop 4
Hop 3
15
and we get the death notice from the first hop
Hop 3
Hop 1
Hop 2
Hop 4
Hop 3
16
Send a packet with a TTL of 2
Hop 3
Hop 1
Hop 2
Hop 4
Hop 3
17
and so on
Hop 3
Hop 1
Hop 2
Hop 4
Hop 3
18
Advantages

• We dont need access (I.e. SNMP) to the routers
• Its very fast
• Standard Internet tool it doesnt break things
• Insignificant load on the routers
• Not likely to show up on IDS reports
• We can probe with many packet types

19
Limitations

• Outgoing paths only
• View is from scanning host only
• Takes a while to collect alternating paths
• Gentle mapping means missed endpoints
• Imputes non-existent links

20
The data can go either way
B
C
D
A
E
F
21
The data can go either way
B
C
D
A
E
F
22
But our test packets only go part of the way
B
C
D
A
E
F
23
We record the hop
B
C
D
A
E
F
24
The next probe happens to go the other way
B
C
D
A
E
F
25
and we record the other hop
B
C
D
A
E
F
26
Weve imputed a link that doesnt exist
B
C
D
A
E
F
27
Remediations

• Alternate routes not a factor on intranets
• Scan from several sources
• stitching needed
• Traceroute in different directions gives
  different interface IP addresses
• Techniques needed to link multiple IP addresses
  to a single host machine

28
Network scanning

• Custom program
• Concurrently scans towards 500 nets at once
• Throttled to 400 packets/sec
• 100 p/s over dialup modems!
• Slow daily scan for host on destination network

29
Data collection complaints

• Australian parliament was the first to complain
• List of whiners (25 nets)
• Military noticed immediately
• Steve Northcutt
• arrangements/warnings to DISA and CERT

30
Visualization goals

• make a map
• show interesting features
• debug our database and collection methods
• hard to fold up
• geography doesnt matter
• use colors to show further meaning

31
(No Transcript)
32
Peacock smashed on a windshield - Dave Presotto

• Interesting art
• tantalizing edges
• interior shows ISPs (colored by IP address!)
• cant trace routes
• cant even find the probe host

33
Colored by AS number
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
Map Coloring

• distance from test host
• IP address
• shows communities
• Geographical (by TLD)
• ISPs
• future
• timing, firewalls, LSRR blocks

52
Colored by IP address!
53
Colored by geography
54
Colored by ISP
55
Colored by distance from scanning host
56
US military reached by ICMP ping
57
US military networks reached by UDP
58
(No Transcript)
59
(No Transcript)
60
Yugoslavia

• An unclassified peek at a new battlefield

61
(No Transcript)
62
Un film par Steve Hollywood Branigan...
63
(No Transcript)
64
fin
65
NYC after 9/11
66
CIDR and IP Counts
67
Routers in New York City
68
Internet before 9/11/2001
69
Internet after 9/11/2001
70
Lets look at some intranets
71
(No Transcript)
72
(No Transcript)
73
(No Transcript)
74
(No Transcript)
75
(No Transcript)
76
(No Transcript)
77
Anything large enough to be called an intranet
isout of control
78
This is not the fault of network administrators!

• Robust internet design frustrates central control
• Ad hoc growth
• Mergers and acquisitions frustrate long-term
  network planning and policies
• CIOs and auditors already know this

79
(No Transcript)
80
(No Transcript)
81
(No Transcript)
82
(No Transcript)
83
We call these routing leaks

• Easily-found holes in the intranet perimeter
• Show up nicely on the maps
• Leaking hosts or routers announce routes to other
  networks or the Internet
• Sometimes left over from an old corporate split
• Non-functional VPNs can show up

84
This was Supposed To be a VPN
85
The maps are useful, but not the main data

• We collect tens of megabytes of network data
• There were unexpected subtleties to this process
• How do you display all this information, given
  that different clients want different data?

86
The second technology host leak detection

• Developed to find hosts that have access to both
  intranet and Internet
• Or across any privilege boundary
• Leaking hosts do not route between the networks
• May be a dual-homed host
• Not always a bad thing
• Technology didnt exist to find these

87
Possible host leaks

• Miss-configured telecommuters connecting remotely
• VPNs that are broken
• DMZ hosts with too much access
• Business partner networks
• Internet connections by rogue managers
• Modem links to ISPs

88
Leak results

• Found home web businesses
• At least two clients have tapped leaks
• One made front page news

89
Leak Detection Prerequisites

• List of potential leakers obtained by census
• Access to intranet
• Simultaneous availability of a mitt

90
Leak Detection Layout
Mapping host
mitt

• Mapping host with address A is connected to the
  intranet
• Mitt with address D has Internet access
• Mapping host and mitt are currently the same
  host, with two interfaces

A
D
Internet
intranet
C
B
Test host
91
Leak Detection
Mapping host
mitt

• Test host has known address B on the intranet
• It was found via census
• We are testing for unauthorized access to the
  Internet, possibly through a different address, C

A
D
Internet
intranet
C
B
Test host
92
Leak Detection
Mapping host
mitt

• A sends packet to B, with spoofed return address
  of D
• If B can, it will reply to D with a response,
  possibly through a different interface

A
D
Internet
intranet
C
B
Test host
93
Leak Detection
Mapping host
mitt

• Packet must be crafted so the response wont be
  permitted through the firewall
• A variety of packet types and responses are used
• Either inside or outside address may be
  discovered
• Packet is labeled so we know where it came from

A
D
Internet
intranet
C
B
Test host
94
Leak Detection
Mapping host
mitt

• This describes outbound leaks
• Inbound leaks are usually much more serious

A
D
Internet
intranet
C
B
Test host
95
Possible problems

• NAT
• egress filtering
• transit of sensitive data over the public networks

96
Our new tools give new views of intranets

• The pictures are mostly for management
• Maps can show progress
• red is bad, blue is good
• we can color the maps in many ways
• The real value in the reports is the list of
  anomalies
• network leaks, routing loops, open routers, etc.

97
How we scan

• Via dialup, using RAS servers
• Secure tunnel, if you prefer
• IP/SEC
• PPTP
• others?

98
What we do

• Probe the network for things not in the official
  list
• Run a host enumeration
• Run leak tests on each host found

99
Technology used

• Traceroute
• SNMP queries
• Router type
• Routing tables
• Pings
• Special leak detection probes
• ICMP
• UDP
• Other possible if requested

100
Report

• HTML-based
• Delivered on CDROM or DVD
• Maps
• Executive summary shows highpoints
• Interactive map viewer tool for Windows

101
Competitors?

• Not yet, not quite
• Many use the same terms, but offer different
  services
• Some components are pretty easy and free
• Host enumeration
• But we do it better (!)
• A bit like HP OpenView
• OpenView doesnt scale
• Much slower

102
Value

• Discovers unknown parts of the network
• Data feeds into existing tools, enhancing their
  value
• You cant secure what you dont know about
• Due diligence for intranets
• Insurance?
• MA activity
• Personnel turnover leaves legacy connections
• Business partners

103
Getting a report

• Web-based
• We can send you a CD-ROM
• You can access a web server
• FreeBSD-based
• One-time password authentication
• Very paranoid server

104
Sample report
105
(No Transcript)
106
(No Transcript)
107
(No Transcript)
108
(No Transcript)
109
Internet report
110
(No Transcript)
111
(No Transcript)
112
(No Transcript)
113
(No Transcript)
114
(No Transcript)
115
(No Transcript)
116
(No Transcript)
117
Intranet Best current practices

• We are acquiring the data to produce a paper
  statistics over a variety of large intranets

118
Some intranet statisticsfrom Lumeta clients
119
(No Transcript)
120
Mapping the Internet and Intranets

• Steve Branigan, Hal Burch, Bill Cheswick
• ches_at_lumeta.com