More 82573L details

1
More 82573L details

• Getting ready to write and test a character-mode
  device-driver for our anchor-LANs ethernet
  controllers

2
A nic.c character driver?
my_isr()
my_fops
ioctl
my_ioctl()
open
my_open()
read
my_read()
write
my_write()
release
my_release()
module_init()
module_exit()
3
Statistics registers

• The 82573L has several dozen statistical counters
  which automatically operate to keep track of
  significant events affecting the ethernet
  controllers performance
• Most are 32-bit read-only registers, and they
  are automatically cleared when read
• Your modules initialization routine could read
  them all (to start counting from zero)

4
Initializing the nics counters

• The statistical counters all have address-
  offsets in the range 0x04000 0x04FFF
• You can use a very simple program-loop to clear
  each of these read-only registers

// Here io is the virtual base-address of the
nics i/o-memory region int r // clear all
of the Pro/1000 controllers statistical
counters for (r 0x4000 r lt 0x4FFF r 4)
ioread32( io r )
5
A few counter examples
0x4000 CRCERRS CRC Errors Count 0x400C
RXERRC Receive Error Count 0x4014 SCC Single
Collision Count 0x4018 ECOL Excessive Collision
Count 0x4074 GPRC Good Packets
Received 0x4078 BPRC Broadcast Packets
Received 0x407C MPRC Multicast Packets
Received 0x40D0 TPR Total Packets
Received 0x40D4 TPT Total Packets
Transmitted 0x40F0 MPTC Multicast Packets
Transmitted 0x40F4 BPTC Broadcast Packets
Transmitted
6
Ethernet packet layout

• Total size normally can vary from 64 bytes up to
  1536 bytes (unless jumbo packets and/or
  undersized packets are enabled)
• The NIC expects a 14-byte packet header and it
  appends a 4-byte CRC check-sum

0 6
12 14
the packets data payload goes here (usually
varies from 56 to 1500 bytes)
destination MAC address (6-bytes)
source MAC address (6-bytes)
Type/length (2-bytes)
Cyclic Redundancy Checksum (4-bytes)
7
Filter registers

• All the modern ethernet controllers have a
  built-in filtering capability which allows the
  NIC to automatically discard any packets having a
  destination-address different from the
  controllers own unique MAC address
• But the 82573L offers a more elaborate filtering
  mechanism (and can also reject packets based on
  the source addresses)

8
How receive works
Buffer0
List of Buffer-Descriptors
descriptor0
descriptor1
Buffer1
descriptor2
descriptor3
0
0
0
Buffer2
0
We setup memory-buffers where we want received
packets to be placed by the NIC We also create
a list of buffer-descriptors and inform the NIC
of its location and size Then, when ready, we
tell the NIC to Go! (i.e., start receiving),
but to let us know when these receptions have
occurred
Buffer3
Random Access Memory
9
Receive Control (0x0100)
31 30 29 28 27 26
25 24 23 22 21
20 19 18 17 16
R 0
0
0
FLXBUF
SE CRC
BSEX
R 0
PMCF
DPF
R 0
CFI
CFI EN
VFE
BSIZE
15 14 13 12 11
10 9 8 7 6 5
4 3 2 1 0
B A M
R 0
MO
DTYP
RDMTS
I L O S
S L U
LPE
UPE
0 0
R 0
SBP
E N
LBM
MPE
EN Receive Enable DTYP Descriptor
Type DPF Discard Pause Frames SBP Store Bad
Packets MO Multicast Offset PMCF Pass MAC
Control Frames UPE Unicast Promiscuous Enable
BAM Broadcast Accept Mode BSEX Buffer Size
Extension MPE Multicast Promiscuous Enable
BSIZE Receive Buffer Size SECRC Strip
Ethernet CRC LPE Long Packet reception Enable
VFE VLAN Filter Enable FLXBUF Flexible
Buffer size LBM Loopback Mode CFIEN
Canonical Form Indicator Enable RDMTS
Rx-Descriptor Minimum Threshold Size CFI
Cannonical Form Indicator bit-value
10
Registers Names

• Memory-information registers
• RDBA(L/H) Receive-Descriptor Base-Address
  Low/High (64-bits)
• RDLEN Receive-Descriptor array Length
• RDH Receive-Descriptor Head
• RDT Receive-Descriptor Tail
• Receive-engine control registers
• RXDCTL Receive-Descriptor Control Register
• RCTL Receive Control Register
• Notification timing registers
• RDTR Receive-interrupt packet Delay Timer
• RADV Receive-interrupt Absolute Delay Value

11
Rx-Desc Ring-Buffer
RDBA base-address
0x00 0x10 0x20 0x30 0x40 0x50 0x60 0x70 0x
80
RDH (head)
RDLEN (in bytes)
RDT (tail)
owned by hardware (nic)
owned by software (cpu)
Circular buffer (128-bytes minimum)
12
Rx-Descriptor Control (0x2828)
31 30 29 28 27 26
25 24 23 22 21
20 19 18 17 16
R 0
R 0
R 0
R 0
R 0
R 0
R 0
G R A N
R 0
R 0
1 --------- 0
ADV D3 WUC
SDP1 DATA --------- 0
SDP0 DATA --------- D/UD status
0
WTHRESH (Writeback Threshold)
15 14 13 12 11
10 9 8 7 6 5
4 3 2 1 0
R 0
R 0
0
FRC DPLX
FRC SPD
0
HTHRESH (Host Threshold)
R 0
R 0
A S D E
0
L R S T
0 0
0
0
PTHRESH (Prefetch Threshold)
Prefetch Threshold A prefetch operation is
considered when the number of valid, but
unprocessed, receive descriptors that the
ethernet controller has in its on-chip buffer
drops below this threshold. Host Threshold - A
prefetch occurs if at least this many valid
descriptors are available in host memory
Writeback Threshold - This field controls the
writing back to host memory of already processed
receive descriptors in the ethernet controllers
on-chip buffer which are ready to be written back
to host memory
GRAN (Granularity) 1descriptor-size,
0cacheline-size
13
Legacy Rx-Descriptor Layout
31


0
0x0 0x4 0x8 0xC
Buffer-Address low (bits 31..0)
Buffer-Address high (bits 63..32)
Packet Checksum
Packet Length (in bytes)
VLAN tag
Status
Errors
Buffer-Address the packet-buffers 64-bit
address in physical memory Packet Length
number of bytes in the data-packet that has was
received Packet Checksum the16-bit
ones-complement of the entire logical packet
Status shows if descriptor has been used and if
its last in a logical packet Errors valid
only when DD and EOP are set in the descriptors
Status field
14
Suggested C syntax

• typedef struct
• unsigned long long base_addr
• unsigned short pkt_length
• unsigned short checksum
• unsigned char desc_stat
• unsigned char desc_errs
• unsigned short vlan_tag
• rx_descriptor

15
RxDesc Status-field
7 6 5 4
3 2 1 0

PIF
IPCS
TCPCS
VP
IXSM
EOP
DD
UDPCS
DD Descriptor Done (1yes, 0no) shows if nic
is finished with descriptor EOP End Of
Packet (1yes, 0no) shows if this packet is
logically last IXSM Ignore Checksum
Indications (1yes, 0no) VP VLAN Packet match
(1yes, 0no) USPCS UDP Checksum calculated in
packet (1yes, 0no) TCPCS TCP Checksum
calculated in packet (1yes, 0no) IPCS IPv4
Checksum calculated on packet (1yes, 0no)
PIF Passed In-exact Filter (1yes, 0no) shows
if software must check
16
RxDesc Error-field
7 6 5 4
3 2 1 0

RXE
IPE
TCPE
reserved 0
SE
CE
SEQ
reserved 0
RXE Received-data Error (1yes, 0no) IPE
IPv4-checksum error TCPE TCP/UDP checksum
error (1yes, 0no) SEQ Sequence error (1yes,
0no) SE Symbol Error (1yes, 0no) CE CRC
Error or alignment error (1yes, 0no)
17
Network Administration

• Some higher-level networking protocols require
  the Operating System to setup a translation
  between the hostname for a workstation and the
  hardware-address of its Network Interface
  Controller
• One mechanism for doing this is creation of a
  specially-named textfile (/etc/ethers) that
  provides database for translations

18
In-class exercise 1

• We put a file named ethers on our course
  website that offers a template for defining the
  translation database that software can consult on
  our anchor clusters LAN
• One of the eight workstations entries has been
  filled in already
• Can you complete this database by adding the MAC
  addresses for the other 7 machines?

0030488A3003 anchor00.cs.usfca.edu
19
Our seereset.c demo

• We created this LKM to demonstrate the sequence
  of state-changes that three of our network
  controllers registers undergo in response to
  initiating a reset operation
• The programming technique used here is one which
  we think could be useful in lots of other
  hardware programming situations where a vendors
  manual may not answer all our questions about how
  devices work

20
In-class exercise 2

• Try redirecting the output from this cat
  command to a file, like this
• cat /proc/seereset gt seereset.out
• Then edit this textfile, adding a comment to each
  line which indicates the bit(s) that experienced
  a change-of-state from the line that came
  before it (thereby providing yourself with a
  running commentary as to how the NIC proceeds
  through a reset)