Networking and Communications

1
Networking and Communications

• David W. Hankins
• 10/28/2008

2
About your Speaker

• Left college early to work for ISP's.
• Yeah, don't do that.
• Now I'm continuing my education.
• Operated IP networks from small dial access to
  large backbones.
• Wrote software for Skycache/Cidera.
• Now a Software Engineer at Internet Systems
  Consortium, Inc. (ISC)?
• Working on the ISC DHCP software project.
• Mostly a maintainer, but also wrote DHCPv6 (DHCP
  for IPv6) software.
• Author, RFC 5071 (don't bother reading it).
• Off-road and video game enthusiast.

3
About ISC

• Internet Systems Consortium, Inc.
• Headquartered in Redwood City, CA
• 501(c)(3) Nonprofit Corporation
• http//www.isc.org/
• Mission
• To develop and maintain production quality Open
  Source software, such as BIND and DHCP
• Enhance the stability of the global DNS through
  reliable F-root nameserver operations and ongoing
  operation of a DNS crisis coordination center,
  ISC's OARC for DNS
• Further protocol development efforts,
  particularly in the areas of DNS evolution and
  facilitating the transition to IPv6.

4
Overview

• The OSI model is actually dead, but you still
  need to know it.
• We'll talk about the historic to current
  progression of Ethernet.
• 'Network' means 'Internet' these days, so I will
  focus there.

5
The OSI Model

• OSI defined 7 Layers in OSI standard
  networking, so that different technologies could
  be used at each layer, and the lower and higher
  layers needed no knowledge.
• This is kind of like Modular Programming for
  networks.
• The 7 OSI Layers (plus Evi Nemeth's extensions)
• 1 Physical ex Twisted Pair
• 2 Link ex Ethernet
• 3 Network ex Ipv4
• 4 Transport ex TCP
• 5 Session
• 6 Presentation
• 7 Application
• 8 Financial
• 9 Political

6
OSI is Dead

• Long live OSI!
• There are still some common uses of these terms.
• In Internet Protocols, the phrase was coined
• IP over Everything. Everything over IP.
• The idea in the Internet's childhood was to
  provide a simple framing format that could be
  used, and communicate with hosts everywhere,
  regardless of the lower layer link protocols (nor
  how proprietary they were). IP over everything.
• It was recognized that once you did this, the
  great bastions of proprietary networks (the telco
  companies) would soon find themselves subverted
  IP could bridge them all.
• Ex Voice over UDP over IP over PPP over SSH over
  TCP over IP over DNS over UDP over IP over
  Ethernet transmitted on twisted pair. Which one
  is layer 3?

7
Ethernet Framing

• Ethernet Network Interface Cards (NICs) were
  assigned (theoretically) unique addresses by
  their manufacturers.
• The first 24 bits are Organizationally Unique
  Identifiers, so for one manufacturer that field
  is fixed, and they assign the remaining bits.
• That didn't work out so well.
• The NIC would filter out any packet it received
  that was not directed to its own address, so the
  OS would not have to discard it. Exception
  broadcast address.
• The 64-octet minimum length was to enforce
  Ethernet timing parameters at 10Mbps, 512 bits
  is 51.2us, so 10km.

8
Ethernet Thicknet

• Ethernet L1's follow a kind of naming convention.
  In the term XXXBase-PHY, X is the speed of the
  link (in megabits per second), and PHY is a
  conventional suffix to describe the link media.
  For example, both 10Base-T and 100Base-T exist.
• When I started working at ISP's, something called
  Thicknet (10Base-5) was just getting phased
  out in favor of Thinnet (10Base-2).
• Thicknet used a coaxial cable, a center conductor
  wire made of copper sheathed in an insulator
  (wax), with a braided second conductor completely
  surrounding it, so that the shield and the center
  pin shared the same axis. A plastic sheath
  enclosed the whole thing.
• The center wire carried signal, the sheath made a
  Farraday Cage.
• At points along the plastic sheath, lines marked
  the points in the wire where Ethernet signals'
  standing waves would stand. Hosts could be
  attached to these points, with two points
  sticking through the sheath and insulator like
  vampires.

9
Ethernet Thinnet

• Note that although the -5 suffix in Thicknet was
  used to describe its maximum distance between
  cable endpoints (500 meters), and the -2 suffix
  used in Thinnet similarly marked the intent for
  it to survive 200 meters, 10Base-2 was later
  clarified to survive no more than 185 meters.
• Thinnet used a coaxial cable (RG-58, smaller than
  Thicknet's RG-8), but rather than vampiric
  connections, the coax cables used BNC connectors
  on the end, and hosts in the network were
  connected on T connections.
• Each end of the coax would be terminated with a
  50 Ohm resistor. This reduced the amplitude of
  reflection signals reaching the cables' ends.
• This was certainly better than Thicknet, but it
  still had problems.

10
Ethernet Do the Twist!

• The 10Base-T standard described the use of
  Category 5 cable. Category 5 cable is a
  performance standard, generally met by using
  twisted pairs of 24 AWG unshielded cables.
• To meet the 16Mhz standard, you might use more
  twists, or different gauge wire.
• Similarly to 10Base-5 and 10Base-2 where the
  center conductor and sheath served as the
  transmit/receive and Ground, twisted pair cable
  usually carries 4 pairs of conductors, where each
  pair is twisted around themselves.
• The twist in the pair averages out noise imparted
  on the line from other signal sources receivers
  look at the voltage difference between the pair,
  not at the absolute voltage of either. By
  twisting the pair, the signal and ground are
  affected near identically, so the difference is
  unaffected by noise.

11
Ethernet RJ45

• The new 10Base-T cables used RJ45 connectors on
  the end, plastic connectors with a retaining
  lever and 8 conductors (for the 4 pair).
• W-O, O, W-G, B, W-B, G, W-B, B
• With 4 pair to choose from, Ethernet was now free
  to use two whole pair for transmit and receive.
• The orange and green pair.
• Ethernet Hubs would present female RJ45's, and
  provided a bus between all the transmit and
  receive pins. This meant that although you had
  separate physical channels for transmit and
  receive, you still had to handle collisions,
  where two nodes transmit at the same time.
• This meant realistically, 10Base could not reach
  10Mbps.

12
Ethernet Duplexing

• Throughout the 10Base-T and well into the
  100Base-TX eras, networks slowly migrated away
  from Ethernet 'Hubs' towards Ethernet 'switches'
  (also referred to as 'bridges').
• A switch's primary difference is that it has its
  own Ethernet receive and transmit chips, which
  receive packets from one ingress interface,
  examine the Ethernet header destination address,
  and select only one egress port to transmit the
  Ethernet packet on (presuming it is not already
  busy).
• Switches slowly build a Forwarding Database by
  observing packets that it receives, and recording
  the source address and the port it received the
  packet on.
• This enabled Ethernet hosts to transmit in Full
  Duplex, at full line speed, receiving and
  transmitting in parallel. No collisions.

13
A day in the Life of your Laptop

• On your OS's desktop, there is some widget that
  lets you pick Wireless-Ethernet networks,
  distinguished by ESSID. This is just Ethernet
  over 802.11(mumble), some form of spread-spectrum
  microwave, with some quirks.
• One quirk is that your NIC will associate with an
  access point, which in essence establishes a
  connection for your NIC with the AP's Ethernet
  broadcast domain. From here down, everything
  looks like Ethernet packets.
• These Ethernet packets will probably be carrying
  a number of things, but we're only interested in
  IP (Internet Protocol) and ARP (Address
  Resolution Protocol) packets.
• ARP provides a way for hosts to find the Ethernet
  MAC address for a given IP address, if it is on
  the same broadcast domain.

14
Getting Configured

• You can't talk to other folks on the Internet
  unless you have an IP address (so they can send
  replies).
• So the first thing your laptop will do once it is
  associated is to start its Dynamic Host
  Configuration Protocol (DHCP) client. The DHCP
  client uses a finite state machine to retain a
  consistent and correct configuration given
  changes in administrative policy (IP renumbering,
  changes in service addresses).
• The client's initial DHCP packets are transmitted
  to the broadcast address, any servers will reply
  to the client's unicast MAC address (with some
  complicated exceptions), offering it an address
  configuration as well as service locators.
• The client selects a configuration and requests
  it.

15
IP Addresses

• IP version 4 addresses are 32-bit fields, usually
  represented by octets 10.1.2.3
• An IP subnet is a specific region of the 32-bit
  space. 10.1.2.0/24, for example, is all those
  addresses where the first 24 bits are 10.1.2,
  and the remaining 8 bits are of any value.
• The IP packet header has lots of fields let's
  just say it at least has a source and destination
  IP address.
• To forward a packet from your laptop to another
  on the same network, your laptop notices
  10.1.2.4 is inside 10.1.2.0/24, and uses ARP
  to unicast directly.
• When you want to talk to someone on another
  network, it gets complicated.

16
IP Routing Basics

• One of the things you got from DHCP was a
  routers option. This lists a number (usually
  one) of IP addresses inside your subnet which you
  should direct packets to get to the rest of the
  world. These are generally referred to as
  default routers.
• Any route is a pair of values a prefix, and a
  destination to forward that prefix to.
• The default route is simply the 0.0.0.0/0 prefix
  directed to the listed default routers' IP
  addresses. Your laptop also carries a
  10.1.2.0/24 route in its table, pointing to your
  NIC.
• To route a packet outward, you start with the
  most specific route(s) in your table, and work
  down to the least specific. The route matches if
  the destination IP address is within the subnet.
  If no route matches, you emit an ICMP error.

17
DNS

• IP addresses are hard to read and type, so we use
  the Domain Name System to map names to resources.
• Now that your laptop is on the network, you start
  up your web browser, and try for
  http//www.isc.org/.
• www.isc.org is not an IP address, it is a
  domain name. To find www.isc.org's IP address,
  your laptop performs a recursive DNS query
  against a nameserver (whose IP address was
  provided by DHCP). The recursive nameserver is
  an assist service extended by your network
  administrator.
• It recursively follows DNS delegations from the
  root nameservers (the silent dot after org), the
  GTLD nameservers (org), and finally their
  delegation to ISC's own nameservers (isc.org),
  which reply with the A record of www.isc.org.
  All these nameservers are referred to as
  'Authoritative' nameservers.

18
UDP and TCP

• DHCP and DNS are protocols that run over UDP
  (over IP over...), although DNS can also be
  carried by TCP.
• They are really just UDP payload data, the UDP
  port essentially directs the packet to a buffer
  to be sent to an application (or discard).
• HTTP, to reach http//www.isc.org/, wants to open
  a TCP connection to the HTTP port 80 on
  www.isc.org. Now that your laptop has this IP
  address in hand, it can do that.
• TCP and UDP differ mainly in that if a UDP packet
  is lost, no one cares (the application must
  retransmit). TCP's whole purpose, however, is to
  make sure a stream of data written in on one end
  reliably reaches the other end, as fast as
  reasonably possible. It tracks RTT's, and
  schedules retransmissions. It negotiates and
  then uses a 'window' of data to completely use
  the network between the nodes. The application
  is unaware of all this.

19
IP Routing Again

• You understand Routing Basics, but how does your
  default router know how to direct packets
  destined to ISC?
• It has its own routing table. It might be
  simple, like yours, using another default route.
  At some point, it will reach a router that is
  running default-less, that has loaded the full
  table of routes for the whole Internet.
• Networks advertise routes for their own address
  space to their customers, peers and transit
  providers using the Border Gateway Protocol.
  These peers then (selectively) extend the
  announcement to their own customers, peers and
  transit providers. This goes around the world.
• A BGP route is again just a prefix, with a
  destination address, and an AS-list to perform
  loop detection. When an IP packet matches a BGP
  destination route, the destination address is
  picked up and re-searched on the routing table.

20
The IGP

• BGP was first laid down to carry those border
  routes the routes external to the network, and
  to advertise least-specifics for the local
  network's address space.
• Recursive lookups on its table must eventually be
  found in the internal network, as a directly
  connected route on the current router.
• So internal or directly attached networks are
  commonly advertised to all the routers in the
  network using ISIS or OSPF IGP Routing
  protocols. These protocols have different design
  and limitations they converge faster and
  support load balancing, but do not scale to large
  numbers of routes.
• Many networks have grown so large, that they
  offload portions of the IGP into BGP itself this
  practice is called iBGP, although the protocol
  is identical.

21
Route Caching

• When a route is finally found, it is worthwhile
  not to repeat all that recursive effort. So, the
  router will insert an entry in a cache.
• There are many caching approaches, the most
  common is Flow Caching, where the IP packet's
  source and destination IP address and TCP/UDP
  ports are combined together to form a unique key
  in the cache (usually a hash table).
• This has a tendency to provide very stable RTT
  times, an advantage over other load-balance
  related caching techniques (like round robin).
• In earlier router architectures, the cache lived
  on the line card holding the interfaces. Today,
  modern routers prepopulate the cache from routing
  information.

22
Internet Growth

• This graph only measures the number of PTR
  records registered in the DNS. The actual number
  of IP addresses in use is probably much higher!
• There are only 232 addresses, but don't worry
  IPv6! Someday...

23
Keeping up with Growth

• The main currency of Internet connectivity is
  bandwidth. Ports for users is just capex.
  Bandwidth charges are forever.
• One trouble is that network interfaces
  classically delivered by Telco companies come
  with fixed monthly costs associated with their
  maximum line rate capacity.
• T1 1.5Mbps, T3 45Mbps, OC-3 150Mbps...
• This creates a stepladder effect in ISP
  operations. Your userbase grows, your T1 is
  getting full. You have to double your expenses
  on a second T1 to support a fraction of a percent
  more customers.
• The ISP profit game was about matching your
  growth and expense curves optimally, timing your
  buildouts in advance of your growth.

24
All your Profits are Belong...

• Maybe you've spotted the flaw in this little
  plan. No matter how much an ISP grows, it just
  gives all its profits to the Telco! Customers
  pay just enough for more bandwidth.
• This continued for a long time, until recent
  years (2000) with the advent of fibre based
  long-haul and metropolitan networks.
• This allowed ISP's to get their own dark fibre
  (or a lambda on a shared fibre using WDM). No
  one can tell you how fast your networking goes
  over light. They just carry your light around.
• So the Telco's just bought the ISP's instead.
• Still, today we bill more in terms of 95ile of
  bandwidth used, and not so much in terms of
  connect rate.

25
Some notes on Telco Lines

• Your home phone is still (unfortunately) analog.
  The Telco does a Digital-To-Analog conversion,
  pushing pulses towards your home carrying the
  audio signal (driving your speaker). When you
  speak, an Analog-To-Digital conversion places
  your mic signal into a digital stream that is
  call-routed outwards.
• This digital stream is 56Kbps of audio, carried
  in a 64Kbps timeslice (8 bits per second are used
  for control).
• These timeslices are MUXed together to form
  telecommunications backbone lines. A T1 is 24 of
  these DS0's. A DS-3 is 30 T1's MUXed together.
  Call routing protocols connect and assign
  timeslices throughout the network.
• ISPs first started by renting fixed allocations
  in these networks (a whole or fractional T1 or T3
  at a time).

26
PSTN Implementation

• The stream of digital audio bits that comprise
  the timeslices inside each DS0, carried in
  whatever level of hierarchical switching, is
  transmitted between nodes as block waves, in its
  digital form using Alternate Mark Inversion
  (AMI).
• Positive or negative voltage is a 1. Neutral is
  a zero. The transmitter produces an equal number
  of positive and negative voltage waves.
• This makes for little if any DC Bias in the DS
  lines.
• One end transmits, using its own internal clock.
  The far end synchs up to the signal as it is
  received, and transmits in the other direction
  using this signal to provide clocking.
• In order to maintain clock synchronization, for
  DS1 a minimum of 1 bit per every eight must be
  set high. In the attempt to send 8 binary
  zeroes, the transmitting node would indicate a
  bipolar violation. Later we used B8ZS and B3ZS.

27
ISDN The Upgrade that Wasn't

• The Integrated Switched Digital Network could
  essentially be understood as a Version 2 of the
  PSTN.
• Basic Rate Interfaces were two pair instead of
  one and delivered two 64Kbps digital channels (in
  AMI form just as before) and one 8Kbps D-channel.
• PRI were carried by DS1, and had 23 64Kbps
  channels and one 64Kbps D-channel.
• You could make digital voice or data calls over
  any of these channels (even opening multiple
  channels to one destination).
• Because it was digital, and also provided a means
  for unadulterated data transmission, Telco
  companies saw it as a value added service.
• So it cost extra to get one. Digital voice calls
  were free just like on a normal phone, but data
  connections were costly.

28
Data over Voice

• So, what you do is you modify your modem's DSP
  firmware.
• When you get a voice call, check to see if it
  passes HDLC framing.
• If it doesn't pass, start modem negotiations. If
  it does pass, push the raw digital bits through
  to your processor for digital PPP negotiation.
• The telco in Washington actually tried to get a
  law passed to make this illegal.

29
Just how Fast

• Well, OC-768 is running these days at about
  40Gbps.
• At IETF 71, Philadelphia, Comcast delivered
  transit for the event via a single 100 Gigabit
  Ethernet line (the prototype).
• Because it was the prototype, the IETF network
  didn't actually have equipment to receive it. So
  the interface towards IETF was actually 10
  individual 10Gig-E's.
• The Ethernet and SONET specifications often do
  what they can to just surpass each other. The
  simple reason is that SONET facilitates ATM
  (Telco profit) whereas Ethernet facilitates IP
  (ISP profit).
• The last DWDM system I saw put 64 lambdas on a
  single fibre.
• So let's say, 6.4 Tbps is approachable.

30
References

• I admit that I lifted the Ethernet framing
  picture from Wikipedia. It appears to be under
  an appropriate license.
• http//en.wikipedia.org/wiki/Ethernet_II_framing
• The Internet Growth image is from ISC's 2008
  domain survey, available on our website (but
  please do ask before including it in
  publications, we'll probably say yes, but we like
  to hear from you anyway).
• http//www.isc.org/ops/ds/
• The rest is all from memory. I make no claims
  any of it is accurate.