Ethernet over SONET

1
Ethernet over SONET

• Dr. John S. Graham
• University of London Computer Centre

2
Ethernet vs SONET
Feature Ethernet SONET
Guaranteed Delivery No (Packet Switched) Yes (Circuit Switched)
Guaranteed BW No Yes
Performance Monitoring CRC Count BER
Protection None (STP?) APS
Fault Location None AIS
Access Costs Low High
3
SONET/SDH Model
4
SONET Network Elements
5
SONET STS-1 Frame
Payload Capacity 756 Bytes 48.384 Mb/s
6
Section Overhead Bytes
7
Line Overhead Bytes
8
Path Overhead Bytes
9
Accurate Bit-Level Timing

• Line Coding
• AMI, AMI RZ
• HDB3
• Block Coding
• 4B/5B (FDDI)
• 5B/6B (E3)
• 8B/10B (Gigabit Ethernet)
• Scrambling
• Frame Synchronous
• Self Synchronous

10
8B/10B Encoding (1/2)

• Ensures equal number of 1s and 0s for
• Clock recovery
• DC Balance
• Detection of single bit transmission errors
• Predefined comma (0011111 and 1100000)
  characters

11
8B/10B Encoding (2/2)

• Data (D) and commands (K) encoded separately
• Octet split into 5 MSB and 3 LSB
• Code groups generated from lookup tables

Decoded Decoded Decoded Decoded Decoded Decoded Decoded Decoded
A B C D E F G H
0 1 0 0 0 0 0 0
2 2 2 2 2 0 0 0
Encoded Encoded Encoded Encoded Encoded Encoded Encoded Encoded Encoded Encoded
A B C D E i F G H j
1 0 1 1 0 1 0 1 0 0
D2.0
12
Control Words (Ordered Sets)
Symbol Encoding Purpose
F K28.5 D21.5 LINK_NOT_AVAILABLE
C K28.5 D10.5 config_reg LINK_CONFIGURATION
I1 K28.5 D5.6 Idle
I2 K28.5 D16.2 Idle
S K27.7 SoF
T K29.7 EoF
R K23.7 CARRIER_EXTEND
H K30.7 ERROR
13
Scrambling in SONET/SDH
SDH STM-1 uses x7 x6 1 polynomial
14
Concatenation
STS-3 155 Mb/s 145.152 Mb/s
STS-3c/VC-4 155 Mb/s 149.76 Mb/s
STS-12 622 Mb/s 580.608 Mb/s
VC-4-4 622 Mb/s 599 Mb/s
STS-12c 622 Mb/s 599.04 Mb/s
15
Provisioning Example
16
Bandwidth Fragmentation
STS-3c
Drop Interface (Node F)
Blocked!
STS-12
Ring
17
Virtual Concatenation
STS-3c
Drop Interface (Node F)
STS-1-3v
STS-12
Ring
18
The Magic of VCAT
Protocol Client Rate Contiguous Concatenation Contiguous Concatenation Virtual Concatenation Virtual Concatenation
Fast Ethernet 100 STS-3c 65 STS-1-2v 98
ESCON 160 STS-12c 26 STS-1-4v 78
Fibre Channel 850 STS-48c 34 STS-3c-6v 92
Gigabit Ethernet 1000 STS-48c 40 STS-3c-7v 92
Fibre Channel 2 1700 STS-48c 66 STS-3c-12v 92
19
Differential Delay
Core Network(Partial Mesh)
Source STS-1-12v
Sink
TX
Ty
20
Multiframe Indicator
Bit 1 Bit 2 Bit 3 Bit 4 Bit 5 Bit 6 Bit 7 Bit 8
MFI2 MSBs (bits 1 - 4) MFI2 MSBs (bits 1 - 4) MFI2 MSBs (bits 1 - 4) MFI2 MSBs (bits 1 - 4) 0 0 0 0
MFI2 LSBs (bits 5 8) MFI2 LSBs (bits 5 8) MFI2 LSBs (bits 5 8) MFI2 LSBs (bits 5 8) 0 0 0 1
CTRL CTRL CTRL CTRL 0 0 1 0
GID GID GID GID 0 0 1 1
0 1 0 0
0 1 0 1
CRC-8 CRC-8 CRC-8 CRC-8 0 1 1 0
CRC-8 CRC-8 CRC-8 CRC-8 0 1 1 1
MST MST MST MST 1 0 0 0
MST MST MST MST 1 0 0 1
1 0 1 0
1 0 1 1
1 1 0 0
1 1 0 1
SQ MSBs (bits 1 4) SQ MSBs (bits 1 4) SQ MSBs (bits 1 4) SQ MSBs (bits 1 4) 1 1 1 0
SQ LSBs (bits 5 8) SQ LSBs (bits 5 8) SQ LSBs (bits 5 8) SQ LSBs (bits 5 8) 1 1 1 1
21
VCAT Example
STS-1-4v
STS-1-3v
STS-12
22
VCAT Puzzle

• An STS-1-2v between Chicago and London
• One STS-1 sent via geostationary satellite
• The other STS-1 sent via transatlantic fibre
• Will it work? -)

23
VCAT Buffer Requirements
VC Path Type Transport Signal Number Paths Buffer Size
STS-1VC-3 STS-12STM-4 12 18 MB
STS-1VC-3 STS-48STM-16 48 71 MB
STS-3cVC-4 STS-12STM-4 4 18 MB
STS-3cVC-4 STS-48STM-16 12 55 MB
24
Link Capacity Adjustment Scheme

• Modify membership of a VCG
• On-demand and hitless bandwidth changes
• Automatic removal of failed VCG members
• Interworking of LCAS-enabled VCG with non-LCAS
  VCG
• Old-style bridge and roll requires double
  bandwidth

25
LCAS Messaging

• Member Status (MST)
• Re-Sequence Acknowledge (RS-Ack)
• Control (CTRL)
• Fixed
• Add
• Norm
• EOS
• Idle
• DNU
• Group ID
• CRC

26
LCAS Signalling Flows (1/3)
MST FAIL (3)
27
LCAS Signalling Flows (2/3)
28
LCAS Signalling Flows (3/3)
CTRL NORM SQ 1
29
Gigabit Ethernet Implementations
30
Gigabit Ethernet (802.3z)
ANSI X3T11 Fibre Channel
IEEE 802.3 Ethernet
IEEE 802.3z Gigabit Ethernet
31
1000BASE-LX/SX
32
Sequence of Events
LINK_NOT_AVAILABLE
LINK_NOT_AVAILABLE

LINK_CONFIGURATION

LINK_CONFIGURATION
IDLE

IDLE
SOP
PREAMBLE
SFD
DA
LLC Data
SA
LENGTH
PADDING
FCS
33
Generic Framing Procedure

• ITU-T G.7041 and ANSI T1.105.02
• Defines mapping for many types of service onto
  SONET/SDH or OTN
• Ethernet, IP/PPP
• GbE, Fibre Channel (inc. DVB), FICON etc
• Excellent bandwidth utilization efficiency
  tailored to suit different client types
• Simple delineation and robust error control
• Extensible

34
Motivation for GFP (1/2)

• ATM
• Cell overhead causes 10 bandwidth inflation
• Adaptation functions needlessly complex
• Packet over SONET (POS)
• Requires all frames to be converted to PPP over
  HDLC
• Byte stuffing causes non-deterministic bandwidth
  inflation
• QoS hard to monitor or guarantee

35
Motivation for GFP (2/2)

• Minimal overhead
• Transport of client PDUs in native format
• Designed for optimized processing
• Easy aggregation of frames from multiple client
  and multiple protocols into shared bandwidth
  channels
• Low latency capabilities for SAN

36
GFP Types of Frame
37
GFP Functional Model
Source IEEE Communications Magazine, May 2002
38
GFP Frame Structure
Source IEEE Communications Magazine, May 2002
39
Payload Types
Value of UPI Field Protocol GFP Mode
0x01 Ethernet F
0x02 PPP (IP MPLS) F
0x03 Fibre Channel T
0x04 FICON T
0x05 ESCON T
0x06 GbE T
0x07 reserved
0x08 MAPOS F
0x09 0xFE reserved
0x00 0xFF unavailable
40
GFP Client-Independent Processes

• Synchronization
• Bit Level
• Frame Delineation
• Scrambling
• Multiplexing
• GFP Frame
• Client PDU

41
GFP Synchronization
42
GFP Frame Delineation
43
GFP Client PDU Multiplexing
Source
Sink
44
GFP-F Encapsulation (1/2)
Ethernet MAC Frame
X 4 to 64
Transmission Order
0 to 65,535 - X
4
1
8
Bit Transmission Order
45
GFP-F Encapsulation (2/2)
46
GFP-T 64B/65B Payload
D1
K1
D2
D3
K2
D4
D5
D6
64B Sequence
000
001
010
011
101
110
111
F
100
Octet Number
47
GFP-T Error Detection

• Leading flag bit is errored
• Error affects Last control-code indicator
• Control-code location address received in error
• Error causes 4-bit control code to be modified

48
GFP-T Superblocks
536-bit Byte-Aligned
49
GFP-T Bandwidth Requirements
Client Signal Client Signal Bandwidth Minimum Transport Channel Size Nominal Transport Channel Bandwidth Number Superblocks per GFP Frame
ESCON 160 Mb/s STS-1-4v VC-3-4v 193.536 Mb/s 1
Fibre Channel 850 Mb/s STS-3c-4vVC-4-6v 898.56 Mb/s 13
Gigabit Ethernet 1000 Mb/s STS-3c-7vVC-4-7v 1.04832 Gb/s 95
50
Pros and Cons GFP-F

• Higher bandwidth efficiency
• Higher Latency
• More buffer memory required
• Core header fields must be calculated

51
Pros and Cons GFP-T

• Supports many protocols
• Low latency
• Ingress core header fields need not be calculated
• Less bandwidth efficient
• More logic required

52
10 G Ethernet 802.3ae
10GBASE-W
10GBASE-R
S 850 nm MM 300 m L 1310 nm SM 10 km E 1550
nm SM 40 km
53
10 GbE Overview

• Usual 802.3 MAC and frame format
• Jumbo frames not included in standard
• Full duplex only
• No shared media
• No CSMA/CD
• Only optical fibre
• No copper interface
• LAN-PHY or WAN-PHY at various reaches

54
The STS-192c WIS Frame
55
Section Line Overhead
Supported by WAN PHY
56
NetherLight Connectivity
57
UKLight NetherLight Today
58
UKLight NetherLight Tomorrow?
Requires that MSPPs are GFP-F enabled
59
Peering Using 10 G WAN-PHY
Chicago
10GBASE-SW
STM-64
10GBASE-SW
London