Register Renaming through Tomasulos Algorithm and Remap Tables

1
Register Renaming through Tomasulos Algorithm
and Remap Tables

• April 28, 2005
• Prof. Nancy Warter-Perez
• Note These lecture notes are modified from
  Profs. Dave Patterson and John Kubiatowicz.

2
Review Dynamic hardware techniques for
out-of-order execution

• HW exploitation of ILP
• Works when cant know dependence at compile time.
• Code for one machine runs well on another
• Scoreboard (ala CDC 6600 in 1963)
• Centralized control structure
• No register renaming, no forwarding
• Pipeline stalls for WAR and WAW hazards.
• Are these fundamental limitations??? (No)
• Reservation stations (ala IBM 360/91 in 1966)
• Distributed control structures
• Implicit renaming of registers (dispatched
  pointers)
• WAR and WAW hazards eliminated by register
  renaming
• Results broadcast to all reservation stations for
  RAW

3
Tomasulo Organization
FP Registers
From Mem
FP Op Queue
Load Buffers
Load1 Load2 Load3 Load4 Load5 Load6
Store Buffers
Add1 Add2 Add3
Mult1 Mult2
Reservation Stations
To Mem
FP adders
FP multipliers
Common Data Bus (CDB)
4
Three Stages of Tomasulo Algorithm

• 1. Issueget instruction from FP Op Queue
• If reservation station free (no structural
  hazard), control issues instr sends operands
  (renames registers).
• 2. Executionoperate on operands (EX)
• When both operands ready then execute if not
  ready, watch Common Data Bus for result
• 3. Write resultfinish execution (WB)
• Write on Common Data Bus to all awaiting units
  mark reservation station available
• Normal data bus data destination (go
  to bus)
• Common data bus data source (come from bus)
• 64 bits of data 4 bits of Functional Unit
  source address
• Write if matches expected Functional Unit
  (produces result)
• Does the broadcast

5
Tomasulo Loop Example

• Loop LD F0 0 R1
• MULTD F4 F0 F2
• SD F4 0 R1
• SUBI R1 R1 8
• BNEZ R1 Loop
• Assume Multiply takes 4 clocks
• Assume first load takes 8 clocks (cache miss),
  second load takes 1 clock (hit)
• To be clear, will show clocks for SUBI, BNEZ
• Reality integer instructions ahead

6
Loop Example
7
Loop Example Cycle 1
8
Loop Example Cycle 2
9
Loop Example Cycle 3

• Implicit renaming sets up DataFlow graph

10
Loop Example Cycle 4

• Dispatching SUBI Instruction

11
Loop Example Cycle 5

• And, BNEZ instruction

12
Loop Example Cycle 6

• Notice that F0 never sees Load from location 80

13
Loop Example Cycle 7

• Register file completely detached from
  computation
• First and Second iteration completely overlapped

14
Loop Example Cycle 8
15
Loop Example Cycle 9

• Load1 completing who is waiting?
• Note Dispatching SUBI

16
Loop Example Cycle 10

• Load2 completing who is waiting?
• Note Dispatching BNEZ

17
Loop Example Cycle 11

• Next load in sequence

18
Loop Example Cycle 12

• Why not issue third multiply?

19
Loop Example Cycle 13
20
Loop Example Cycle 14

• Mult1 completing. Who is waiting?

21
Loop Example Cycle 15

• Mult2 completing. Who is waiting?

22
Loop Example Cycle 16
23
Loop Example Cycle 17
24
Loop Example Cycle 18
25
Loop Example Cycle 19
26
Loop Example Cycle 20
27
Why can Tomasulo overlap iterations of loops?

• Register renaming
• Multiple iterations use different physical
  destinations for registers (dynamic loop
  unrolling).
• Reservation stations
• Permit instruction issue to advance past integer
  control flow operations
• Other idea Tomasulo building DataFlow graph on
  the fly.

28
What about Precise Interrupts?

• Both Scoreboard and Tomasulo haveIn-order
  issue, out-of-order execution, and out-of-order
  completion
• Need to fix the out-of-order completion aspect
  so that we can find precise breakpoint in
  instruction stream.

29
Relationship between precise interrupts and
specultation

• Speculation is a form of guessing.
• Important for branch prediction
• Need to take our best shot at predicting branch
  direction.
• If we issue multiple instructions per cycle, lose
  lots of potential instructions otherwise
• Consider 4 instructions per cycle
• If take single cycle to decide on branch, waste
  from 4 - 7 instruction slots!
• If we speculate and are wrong, need to back up
  and restart execution to point at which we
  predicted incorrectly
• This is exactly same as precise exceptions!
• Technique for both precise interrupts/exceptions
  and speculation in-order completion or commit

30
HW support for precise interrupts

• Need HW buffer for results of uncommitted
  instructions reorder buffer
• 3 fields instr, destination, value
• Reorder buffer can be operand source gt more
  registers like RS
• Use reorder buffer number instead of reservation
  station when execution completes
• Supplies operands between execution complete
  commit
• Once operand commits, result is put into
  register
• Instructionscommit
• As a result, its easy to undo speculated
  instructions on mispredicted branches or on
  exceptions

31
Four Steps of Speculative Tomasulo Algorithm

• 1. Issueget instruction from FP Op Queue
• If reservation station and reorder buffer slot
  free, issue instr send operands reorder
  buffer no. for destination (this stage sometimes
  called dispatch)
• 2. Executionoperate on operands (EX)
• When both operands ready then execute if not
  ready, watch CDB for result when both in
  reservation station, execute checks RAW
  (sometimes called issue)
• 3. Write resultfinish execution (WB)
• Write on Common Data Bus to all awaiting FUs
  reorder buffer mark reservation station
  available.
• 4. Commitupdate register with reorder result
• When instr. at head of reorder buffer result
  present, update register with result (or store to
  memory) and remove instr from reorder buffer.
  Mispredicted branch flushes reorder buffer
  (sometimes called graduation)

32
What are the hardware complexities with reorder
buffer?

• Need fully-associative comparison with all
  reorder entries on instruction dispatch to find
  latest version of register.
• Need as many ports as register file

33
Tomasulo With Reorder buffer
Done?
FP Op Queue
ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1
Newest
Reorder Buffer
Oldest
F0
LD F0,10(R2)
N
Registers
To Memory
Dest
from Memory
Dest
Dest
Reservation Stations
FP adders
FP multipliers
34
Tomasulo With Reorder buffer
Done?
FP Op Queue
ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1
Newest
Reorder Buffer
Oldest
Registers
To Memory
Dest
from Memory
Dest
2 ADDD R(F4),ROB1
Dest
Reservation Stations
FP adders
FP multipliers
35
Tomasulo With Reorder buffer
Done?
FP Op Queue
ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1
Newest
Reorder Buffer
Oldest
Registers
To Memory
Dest
from Memory
Dest
2 ADDD R(F4),ROB1
6 ADDD ROB5, R(F6)
Dest
Reservation Stations
1 10R2
5 0R3
FP adders
FP multipliers
36
Tomasulo With Reorder buffer
Done?
FP Op Queue
ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1
Newest
Reorder Buffer
Oldest
Registers
To Memory
Dest
from Memory
Dest
2 ADDD R(F4),ROB1
6 ADDD ROB5, R(F6)
Dest
Reservation Stations
FP adders
FP multipliers
37
Tomasulo With Reorder buffer
Done?
FP Op Queue
ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1
Newest
Reorder Buffer
Oldest
Registers
To Memory
Dest
from Memory
Dest
2 ADDD R(F4),ROB1
Dest
Reservation Stations
FP adders
FP multipliers
38
Tomasulo With Reorder buffer
Done?
FP Op Queue
ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1
Newest
Reorder Buffer
F2
DIVD F2,F10,F6
N
F10
ADDD F10,F4,F0
N
Oldest
F0
LD F0,10(R2)
N
Registers
To Memory
Dest
from Memory
Dest
2 ADDD R(F4),ROB1
Dest
Reservation Stations
FP adders
FP multipliers
39
Memory DisambiguationSorting out RAW Hazards in
memory

• Question Given a load that follows a store in
  program order, are the two related?
• (Alternatively is there a RAW hazard between the
  store and the load)? Eg st 0(R2),R5
  ld R6,0(R3)
• Can we go ahead and start the load early?
• Store address could be delayed for a long time by
  some calculation that leads to R2 (divide?).
• We might want to issue/begin execution of both
  operations in same cycle.
• Today Answer is that we are not allowed to start
  load until we know that address 0(R2) ? 0(R3)
• Next Week We might guess at whether or not they
  are dependent (called dependence speculation)
  and use reorder buffer to fixup if we are wrong.

40
Hardware Support for Memory Disambiguation

• Need buffer to keep track of all outstanding
  stores to memory, in program order.
• Keep track of address (when becomes available)
  and value (when becomes available)
• FIFO ordering will retire stores from this
  buffer in program order
• When issuing a load, record current head of store
  queue (know which stores are ahead of you).
• When have address for load, check store queue
• If any store prior to load is waiting for its
  address, stall load.
• If load address matches earlier store address
  (associative lookup), then we have a
  memory-induced RAW hazard
• store value available ? return value
• store value not available ? return ROB number of
  source
• Otherwise, send out request to memory
• Actual stores commit in order, so no worry about
  WAR/WAW hazards through memory.

41
Memory Disambiguation
Done?
FP Op Queue
ROB7 ROB6 ROB5 ROB4 ROB3 ROB2 ROB1
Newest
F4
RF5
LD F4, 10(R3)
N
Reorder Buffer
--
RF5
ST 10(R3), F5
N
F0
LD F0,32(R2)
N
Oldest
--
ltval 1gt
ST 0(R3), F4
Y
Registers
To Memory
Dest
from Memory
Dest
Dest
Reservation Stations
2 32R2
FP adders
FP multipliers
42
Explicit Register Renaming

• Make use of a physical register file that is
  larger than number of registers specified by ISA
• Keep a translation table
• ISA register gt physical register mapping
• When register is written, replace table entry
  with new register from freelist.
• Physical register becomes free when not being
  used by any instructions in progress.
• Pipeline can be exactly like standard DLX
  pipeline
• IF, ID, EX, etc.
• Advantages
• Removes all WAR and WAW hazards
• Like Tomasulo, good for allowing full
  out-of-order completion
• Allows data to be fetched from a single register
  file
• Makes speculative execution/precise interrupts
  easier
• All that needs to be undone for precise break
  pointis to undo the table mappings

43
Explicit register renamingHardware equivalent
of static, single-assignment (SSA) compiler form
Current Map Table
Freelist

• Physical register file larger than ISA register
  file
• On issue, each instruction that modifies a
  register is allocated new physical register from
  freelist
• Used on R10000, Alpha 21264, HP PA8000

44
Explicit register renamingHardware equivalent
of static, single-assignment (SSA) compiler form
Done?
Current Map Table
Freelist
F0
P0
LD P32,10(R2)
N

• Note that physical register P0 is dead (or not
  live) past the point of this load.
• When we go to commit the load, we free up

45
Explicit register renamingHardware equivalent
of static, single-assignment (SSA) compiler form
Done?
Current Map Table
F10
P10
ADDD P34,P4,P32
N
Freelist
F0
P0
LD P32,10(R2)
N
46
Explicit register renamingHardware equivalent
of static, single-assignment (SSA) compiler form
Current Map Table
Freelist
?
Checkpoint at BNE instruction
P60
P62
47
Explicit register renamingHardware equivalent
of static, single-assignment (SSA) compiler form
Done?
Current Map Table
--
ST 0(R3),P40
Y
F0
P32
ADDD P40,P38,P6
Y
F4
P4
LD P38,0(R3)
Y
--
BNE P36,ltgt
N
F2
P2
DIVD P36,P34,P6
N
F10
P10
ADDD P34,P4,P32
y
Freelist
F0
P0
LD P32,10(R2)
y
?
Checkpoint at BNE instruction
P60
P62
48
Explicit register renamingHardware equivalent
of static, single-assignment (SSA) compiler form
Done?
Current Map Table
F2
P2
DIVD P36,P34,P6
N
F10
P10
ADDD P34,P4,P32
y
Freelist
F0
P0
LD P32,10(R2)
y
Speculation error fixed by restoring map table
and freelist
?
Checkpoint at BNE instruction
P60
P62
49
QuestionCan we use explicit register renaming
with scoreboard?
Rename Table
50
Scoreboard Example

• Initialized Rename Table

51
Renamed Scoreboard 1

• Each instruction allocates free register
• Similar to single-assignment compiler
  transformation

52
Renamed Scoreboard 2
53
Renamed Scoreboard 3
54
Renamed Scoreboard 4
55
Renamed Scoreboard 5
56
Renamed Scoreboard 6
57
Renamed Scoreboard 7
58
Renamed Scoreboard 8
59
Renamed Scoreboard 9
60
Renamed Scoreboard 10

• Notice that P32 not listed in Rename Table
• Still live. Must not be reallocated by accident

61
Renamed Scoreboard 11
62
Renamed Scoreboard 12
63
Renamed Scoreboard 13
64
Renamed Scoreboard 14
65
Renamed Scoreboard 15
66
Renamed Scoreboard 16
67
Renamed Scoreboard 17
68
Renamed Scoreboard 18
69
Explicit Renaming Support Includes

• Rapid access to a table of translations
• A physical register file that has more registers
  than specified by the ISA
• Ability to figure out which physical registers
  are free.
• No free registers ? stall on issue
• Thus, register renaming doesnt require
  reservation stations. However
• Many modern architectures use explicit register
  renaming Tomasulo-like reservation stations to
  control execution.

70
Dynamic Scheduling in PowerPC 604 and Pentium Pro

• Both In-order Issue, Out-of-order execution,
  In-order Commit
• Pentium Pro more like a scoreboard since central
  control vs. distributed

71
Dynamic Scheduling in PowerPC 604 and Pentium Pro

• Parameter PPC PPro
• Max. instructions issued/clock 4 3
• Max. instr. complete exec./clock 6 5
• Max. instr. commited/clock 6 3
• Window (Instrs in reorder buffer) 16 40
• Number of reservations stations 12 20
• Number of rename registers 8int/12FP 40
• No. integer functional units (FUs) 2 2No.
  floating point FUs 1 1 No. branch FUs 1 1 No.
  complex integer FUs 1 0No. memory FUs 1 1 load
  1 store

Q How pipeline 1 to 17 byte x86 instructions?
72
Dynamic Scheduling in Pentium Pro

• PPro doesnt pipeline 80x86 instructions
• PPro decode unit translates the Intel
  instructions into 72-bit micro-operations (
  DLX)
• Sends micro-operations to reorder buffer
  reservation stations
• Takes 1 clock cycle to determine length of 80x86
  instructions 2 more to create the
  micro-operations
• 12-14 clocks in total pipeline ( 3 state
  machines)
• Many instructions translate to 1 to 4
  micro-operations
• Complex 80x86 instructions are executed by a
  conventional microprogram (8K x 72 bits) that
  issues long sequences of micro-operations

73
Summary

• Dynamic hardware schemes can unroll loops
  dynamically in hardware
• Form of limited dataflow
• Explicit Renaming more physical registers than
  needed by ISA.
• Rename table tracks current association between
  architectural registers and physical registers
• Uses a translation table to perform compiler-like
  transformation on the fly
• Precise exceptions/Speculation Out-of-order
  execution, In-order commit (reorder buffer)