CS 213 Introduction to Computer Systems Thinking Inside the Box

1
CS 213Introduction to Computer SystemsThinking
Inside the Box

• Instructor Brian M. Dennis
• bmd_at_cs.northwestern.edu
• Teaching Assistant Bin Lin
• binlin_at_cs.northwestern.edu

2
Todays Topics

• Quick Binary Arithmetic Recap
• W/Examples
• Bit Vector Operations
• Byte Oriented Memory
• Floating Point Intro

• UNIX check

3
Numeric Representations

• Base 10 Number Representation
• Decimal 1521310
• 3 x 10 0 1 x 101 2 x 102 5 x 103 1 x
  104
• Base 2 Number Representation
• Represent 1521310 as 111011011011012
• Represent 1.2010 as 1.001100110011001100112
• Represent 1.5213 X 104 as 1.11011011011012 X 213
• Electronic Implementation eased by binary
  representations
• Easy to store with bistable elements
• Reliably transmitted on noisy and inaccurate
  wires
• Straightforward implementation of arithmetic
  functions

4
Arithmetic Example

• Think about how to deal with subtraction
• 2766 decimal
• 0xace
• convert to binary
• 2 16 bit strings
• 0xace
• 0xbeef

• Unsigned addition
• Two's complement addition

5
The Flip 1 trick

• Negating 2's complement number
• Flip bits 1

6
Visualizing Unsigned Addition
Overflow
UAdd4(u , v)

• Wraps Around
• If true sum 2w
• At most once

True Sum
Overflow
v
Modular Sum
u
7
Characterizing TAdd

• Functionality
• True sum requires w1 bits
• Drop off MSB
• Treat remaining bits as 2s comp. integer

PosOver
Case 4
Case 3
Case 2
Case 1
NegOver
(NegOver)
(PosOver)
8
Mathematical Properties of TAdd

• Isomorphic Algebra to UAdd
• TAddw(u , v) U2T(UAddw(T2U(u ), T2U(v)))
• Since both have identical bit patterns
• Twos Complement Under TAdd Forms a Group
• Closed, Commutative, Associative, 0 is additive
  identity
• Every element has additive inverse
• Let TCompw (u )    U2T(UCompw(T2U(u ))
• TAddw(u , TCompw (u ))    0

9
Bit-Level Operations in C

• Operations , , , Available in C
• Apply to any integral data type
• long, int, short, char
• View arguments as bit vectors
• Arguments applied bit-wise
• Examples (Char data type)
• 0x41 --gt 0xBE
• 010000012 --gt 101111102
• 0x00 --gt 0xFF
• 000000002 --gt 111111112
• 0x69 0x55 --gt 0x41
• 011010012 010101012 --gt 010000012
• 0x69 0x55 --gt 0x7D
• 011010012 010101012 --gt 011111012

10
Contrast Logic Operations in C

• Contrast to Logical Operators
• , , !
• View 0 as False
• Anything nonzero as True
• Always return 0 or 1
• Early termination
• Examples (char data type)
• !0x41 --gt 0x00
• !0x00 --gt 0x01
• !!0x41 --gt 0x01
• 0x69 0x55 --gt 0x01
• 0x69 0x55 --gt 0x01
• p p (avoids null pointer access)

11
Shift Operations
01100010
Argument x

• Left Shift x ltlt y
• Shift bit-vector x left y positions
• Throw away extra bits on left
• Fill with 0s on right
• Right Shift x gtgt y
• Shift bit-vector x right y positions
• Throw away extra bits on right
• Logical shift
• Fill with 0s on left
• Arithmetic shift
• Replicate most significant bit on right
• Useful with twos complement integer
  representation

00010000
ltlt 3
00010000
00010000
00011000
Log. gtgt 2
00011000
00011000
00011000
Arith. gtgt 2
00011000
00011000
10100010
Argument x
00010000
ltlt 3
00010000
00010000
00101000
Log. gtgt 2
00101000
00101000
11101000
Arith. gtgt 2
11101000
11101000
12
Bit Shift Examples

• CSAPP Practice 2.15

13
You Should Be Datalab Ready

• Boolean Ops
• Unsigned Arith
• 2's Complement Arith
• Logical Ops

• UNIX Check

14
P4 Chip
15
A Higher LevelPicture
CPU
Register file
ALU
PC
System bus
Memory bus
Main memory
I/O bridge
Bus interface
I/O bus
Expansion slots for other devices such as network
adapters
USB controller
Disk controller
Graphics adapter
Mouse
Keyboard
Display
Disk
hello executable stored on disk
16
Byte-Oriented Memory Organization

• Programs Refer to Virtual Addresses
• Conceptually very large array of bytes
• Actually implemented with hierarchy of different
  memory types
• SRAM, DRAM, disk
• Only allocate for regions actually used by
  program
• In Unix and Windows NT, address space private to
  particular process
• Program being executed
• Program can clobber its own data, but not that of
  others

• Compiler Run-Time System Control Allocation
• Where different program objects should be stored
• Multiple mechanisms static, stack, and heap
• In any case, all allocation within single virtual
  address space

17
Machine Words

• Machine Has Word Size
• Nominal size of integer-valued data
• Including addresses
• Most current machines are 32 bits (4 bytes)
• Limits addresses to 4GB
• Becoming too small for memory-intensive
  applications
• High-end systems are 64 bits (8 bytes)
• Potentially address ? 1.8 X 1019 bytes
• Machines support multiple data formats
• Fractions or multiples of word size
• Always integral number of bytes

18
Word-Oriented Memory Organization
64-bit Words
32-bit Words
Bytes
Addr.
0000
Addr ??
0001
0002
0000
Addr ??
0003

• Addresses Specify Byte Locations
• Address of first byte in word
• Addresses of successive words differ by 4
  (32-bit) or 8 (64-bit)

0004
0000
Addr ??
0005
0006
0004
0007
0008
Addr ??
0009
0010
0008
Addr ??
0011
0012
0008
Addr ??
0013
0014
0012
0015
19
Data Representations
20
Byte Ordering

• How should bytes within multi-byte word be
  ordered in memory?
• Conventions
• Suns, Macs are Big Endian machines
• Least significant byte has highest address
• Alphas, PCs are Little Endian machines
• Least significant byte has lowest address

21
Byte Ordering Example

• Big Endian
• Least significant byte has highest address
• Little Endian
• Least significant byte has lowest address
• Example
• Variable x has 4-byte representation 0x01234567
• Address given by x is 0x100

Big Endian
01
23
45
67
Little Endian
67
45
23
01
22
Implications of Byte Ordering

• Network Byte Ordering

• The Point
• Know what someone means when they're talking
  about big vs little endian

23
Who Is This Man?
William Kahan 1989 ACM Turing Ward Winner
Primary inventor and designer of IEEE 754
floating point standard
24
IEEE Floating Point

• IEEE Standard 754
• Established in 1985 as uniform standard for
  floating point arithmetic
• Before that, many idiosyncratic formats
• Supported by all major CPUs
• Driven by Numerical Concerns
• Nice standards for rounding, overflow, underflow
• Hard to make go fast
• Numerical analysts predominated over hardware
  types in defining standard

25
Fractional Binary Numbers
2i

• Representation
• Bits to right of binary point represent
  fractional powers of 2
• Represents rational number

2i1
4

2
1
1/2

1/4
1/8
2j
26
Frac. Binary Number Examples

• Value Representation
• 5-3/4 101.112
• 2-7/8 10.1112
• 63/64 0.1111112
• Observations
• Divide by 2 by shifting right
• Multiply by 2 by shifting left
• Numbers of form 0.1111112 just below 1.0
• 1/2 1/4 1/8 1/2i ? 1.0
• Use notation 1.0 ?

27
Representable Numbers

• Limitation
• Can only exactly represent numbers of the form
  x/2k
• Other numbers have repeating bit representations,
  cant represent
• Value Representation
• 1/3 0.0101010101012
• 1/5 0.00110011001100112
• 1/10 0.000110011001100112

28
Floating Point Representation

• Numerical Form (Scientific Notation Essentially)
• 1s M 2E
• Sign bit s determines whether number is negative
  or positive
• Significand M normally a fractional value in
  range 1.0,2.0).
• Exponent E weights value by power of two
• Encoding (Not Quite Scientific Notation)
• MSB is sign bit
• exp field encodes E
• frac field encodes M

s
exp
frac
29
Floating Point Precisions

• Encoding
• MSB is sign bit
• exp field encodes E
• frac field encodes M
• Sizes (IEEE Standard)
• Single precision 8 exp bits, 23 frac bits 32
  bits total
• Double precision 11 exp bits, 52 frac bits 64
  bits total
• Extended precision 15 exp bits, 63 frac bits
  80 bits total
• (super whizzy Intel version)

30
Encoding Interpretation

• 3 Cases
• Normalized 's
• exp, not all 0's not all 1's
• Denormalized 's
• exp, all 0's
• Special Values
• exp, all 1's

31
Normalized Numeric Values

• Condition
•  exp ? 0000 and exp ? 1111
• Exponent coded as biased value
•  E Exp Bias
• Exp unsigned value denoted by exp
• Bias Bias value
• Single precision 127 (Exp 1254, E -126127)
• Double precision 1023 (Exp 12046, E
  -10221023)
• in general Bias 2e-1 - 1, where e is number of
  exponent bits
• Significand coded with implied leading 1
•  M 1.xxxx2
•  xxxx bits of frac
• Minimum when 0000 (M 1.0)
• Maximum when 1111 (M 2.0 ?)
• Get extra leading bit for free

32
Normalized Encoding Example

• Value
• Float F 15213.0
• 1521310 111011011011012 1.11011011011012 X
  213
• Significand
• M 1.11011011011012
• frac 110110110110100000000002
• Exponent
• E 13
• Bias 127
• Exp 140 100011002

Floating Point Representation Hex 4 6
6 D B 4 0 0 Binary 0100 0110
0110 1101 1011 0100 0000 0000 140 100 0110
0 15213 1110 1101 1011 01
33
Denormalized Values

• Condition
•  exp 0000
• Value
• Exponent value E Bias 1
• Significand value M 0.xxxx2
• xxxx bits of frac
• Cases
• exp 0000, frac 0000
• Represents value 0
• Note that have distinct values 0 and 0
• exp 0000, frac ? 0000
• Numbers very close to 0.0
• Lose precision as get smaller
• Gradual underflow

34
Special Values

• Condition
•  exp 1111
• Cases
• exp 1111, frac 0000
• Represents value???(infinity)
• Operation that overflows
• Both positive and negative
• E.g., 1.0/0.0 ?1.0/?0.0 ?, 1.0/?0.0 ??
• exp 1111, frac ? 0000
• Not-a-Number (NaN)
• Represents case when no numeric value can be
  determined
• E.g., sqrt(1), ?????

35
Tiny Floating Point Example

• 8-bit Floating Point Representation
• the sign bit is in the most significant bit.
• the next four bits are the exponent, with a bias
  of 7.
• the last three bits are the frac
• Same General Form as IEEE Format
• normalized, denormalized
• representation of 0, NaN, infinity

0
2
3
6
7
s
exp
frac
36
Values Related to the Exponent
Exp exp E 2E 0 0000 -6 1/64 (denorms) 1 0001 -6
1/64 2 0010 -5 1/32 3 0011 -4 1/16 4 0100 -3 1/8 5
0101 -2 1/4 6 0110 -1 1/2 7 0111
0 1 8 1000 1 2 9 1001 2 4 10 1010 3 8 11 1011
4 16 12 1100 5 32 13 1101 6 64 14 1110 7 128 15
1111 n/a (inf, NaN)
37
Dynamic Range
s exp frac E Value 0 0000 000 -6 0 0 0000
001 -6 1/81/64 1/512 0 0000 010 -6 2/81/64
2/512 0 0000 110 -6 6/81/64 6/512 0 0000
111 -6 7/81/64 7/512 0 0001 000 -6 8/81/64
8/512 0 0001 001 -6 9/81/64 9/512 0 0110
110 -1 14/81/2 14/16 0 0110 111 -1 15/81/2
15/16 0 0111 000 0 8/81 1 0 0111
001 0 9/81 9/8 0 0111 010 0 10/81
10/8 0 1110 110 7 14/8128 224 0 1110
111 7 15/8128 240 0 1111 000 n/a inf
closest to zero
Denormalized numbers
largest denorm
smallest norm
closest to 1 below
Normalized numbers
closest to 1 above
largest norm
38
Summary

• Unsigned 2's Complement Arithmetic
• Know it
• C bit operations
• Know em'
• IEEE Floating Point
• Know it
• More on Monday
• Byte Oriented Machine Arch
• Have a sense

• Reading
• Through 2.3

39
Multiplication

• Computing Exact Product of w-bit numbers x, y
• Either signed or unsigned
• Ranges
• Unsigned 0 x y (2w 1) 2 22w 2w1
  1
• Up to 2w bits
• Twos complement min x y (2w1)(2w11)
  22w2 2w1
• Up to 2w1 bits
• Twos complement max x y (2w1) 2 22w2
• Up to 2w bits, but only for (TMinw)2
• Maintaining Exact Results
• Would need to keep expanding word size with each
  product computed
• Done in software by arbitrary precision
  arithmetic packages

40
Unsigned Multiplication in C
u
Operands w bits
v

u v
True Product 2w bits
UMultw(u , v)
Discard w bits w bits

• Standard Multiplication Function
• Ignores high order w bits
• Implements Modular Arithmetic
• UMultw(u , v) u v mod 2w

41
Unsigned vs. Signed Multiplication

• Unsigned Multiplication
• unsigned ux (unsigned) x
• unsigned uy (unsigned) y
• unsigned up ux uy
• Truncates product to w-bit number up
  UMultw(ux, uy)
• Modular arithmetic up ux ? uy mod 2w
• Twos Complement Multiplication
• int x, y
• int p x y
• Compute exact product of two w-bit numbers x, y
• Truncate result to w-bit number p TMultw(x, y)
• Relation
• Signed multiplication gives same bit-level result
  as unsigned
• up (unsigned) p

42
Power-of-2 Multiply with Shift

• Operation
• u ltlt k gives u 2k
• Both signed and unsigned
• Examples
• u ltlt 3 u 8
• u ltlt 5 - u ltlt 3 u 24
• Most machines shift and add much faster than
  multiply
• Compiler can generate this code automatically

k
u
  
Operands w bits
2k

0
0
1
0
0
0


u 2k
True Product wk bits
0
0
0

UMultw(u , 2k)
0
0
0


Discard k bits w bits
TMultw(u , 2k)
43
Unsigned Power-of-2 Divide with Shift

• Quotient of Unsigned by Power of 2
• u gtgt k gives ? u / 2k ?
• Uses logical shift

k
u
Binary Point

Operands
2k
/
0
0
1
0
0
0


u / 2k
Division
.

0

Result
? u / 2k ?

0

44
Signed Power-of-2 Divide with Shift

• Quotient of Signed by Power of 2
• x gtgt k gives ? x / 2k ?
• Uses arithmetic shift
• Rounds wrong direction when u lt 0

45
Correct Power-of-2 Divide

• Quotient of Negative Number by Power of 2
• Want ? x / 2k ? (Round Toward 0)
• Compute as ? (x2k-1)/ 2k ?
• In C (x (1ltltk)-1) gtgt k
• Biases dividend toward 0
• Case 1 No rounding

k
Dividend
u
1

0
0
0

2k 1
0
0
0
1
1
1


Binary Point
1

1
1
1

Divisor
2k
/
0
0
1
0
0
0


? u / 2k ?
.
1

0
1
1

1
1
1
1

Biasing has no effect
46
Correct Power-of-2 Divide (Cont.)
Case 2 Rounding
k
Dividend
x
1


2k 1
0
0
0
1
1
1


1


Binary Point
Incremented by 1
Divisor
2k
/
0
0
1
0
0
0


? x / 2k ?
.
1

0
1
1

1

Biasing adds 1 to final result
Incremented by 1