Automatic Floating-Point to Fixed-Point Transformations

1
Automatic Floating-Point to Fixed-Point
Transformations

• Kyungtae Han,
• Alex G. Olson,
• Brian L. Evans
• Dept. of Electrical and Computer Engineering
• The University of Texas at Austin

2006 Asilomar Conference on Signals, Systems,
and Computers
October 30th, 2006
2
Outline

• Introduction
• Background
• Fixed-point wordlength optimizations
• Automate transformations of systems
• Conclusion

3
Implementing Digital Signal Processing Algorithms
Price
Power
Hardware
Floating- Point Processor

Floating-Point Program
Code Conversion
Digital Signal Processing Algorithms
Fixed- Point Processor
Fixed Point (Uniform Wordlength)

Wordlength Optimization
L
H
Fixed- Point ASIC
Fixed Point (Optimized Wordlength)

L
H
ASIC Application Specific Integrated Circuit
Power consumption
4
Transformations to Fixed Point

• Advantages
• Lower hardware complexity
• Lower power consumption
• Faster speed in processing
• Disadvantages
• Introduces distortion due toquantization error
• Search for optimum wordlengthby trial error is
  time-consuming
• Research goals
• Automate transformations to fixed point
• Control distortion vs. complexity tradeoffs

Floating-Point Program
Code Conversion
Transformation
Wordlength Optimization
Fixed Point (Optimized Wordlength)
5
Distortion vs. Complexity Tradeoffs

• Shorter wordlength may increase application
  distortion and decrease implementation complexity

Applicationdistortion d(w)
c(w) Implementation cost function
d(w) Application distortion function
Feasible region
Optimal tradeoff curve
Implementation complexity c(w)

• Minimize implementation cost
• Minimize application distortion

6
Search for Optimum Wordlength

• Complete search
• Search whole space
• Impractical in systems with many variables
• Gradient-based search
• Utilizes gradient information to determine next
  candidates
• Complexity measure (CM) Sung and Kum, 1995
• Distortion measure (DM) Han et al., 2001
• Complexity-and-distortion measure (CDM) Han and
  Evans, 2004
• Guided random search
• Genetic algorithm for single objective Leban and
  Tasic, 2000
• Multiple objective genetic algorithm

7
Complexity-and-Distortion Measure

• Weighted combination of measures
• Single objective function
• Gradient-based search
• Initialization
• Iterative greedy search based on complexity and
  distortiongradient information

c(w) Complexity function
d(w) Distortion function
Dmax Constant for maximum distortion
Cmax Constant for maximum complexity
Wordlength lower bound
Wordlength upper bound
8
Genetic Algorithm

• Evolutionary algorithm
• Inspired by Holland 1975
• Mimic processes of plant and animal evolution
• Find optimum of a complex function

From Greg Rohlings Ph.D Defense 2004
9
Case Study Filter Design

• Infinite impulse response (IIR) filter
• Complexity measure Area model of
  field-programmable gate array (FPGA)
  Constantinides, Cheung, and Luk 2003
• Distortion measure Root mean square (RMS) error
• Seven fixed-point variables (indicated by slashes)

10
Case Study Gradient-Based Search

• CDM could lead to lower complexity and lower
  number of simulations compared to DM and CM

Search Method Gradient Measure Number of Simulations Complexity Estimate (LUT) Distortion (RMS)
Gradient Gradient Gradient Complete DM CDM CM - 316 145 417 167 51.05 49.85 51.95 - 0.0981 0.0992 0.0986 -
Maximum distortion measured by root mean square
(RMS) error is 0.1 167 268,435,456 (8.5
years, if 1 second per 1 simulation)
11
Case Study - IIR Genetic Algorithm

• Search Pareto optimal set (nondominated)
• Handles multiple objectives Error and Area

Pareto Front
22,500 simulations
45,000 simulations
9,000 simulations
100th Generation
250th Generation
500th Generation
Population for one generation 90
LUT Lookup table
12
Case Study Comparison
Contribution 1

• Superpose gradient-based search (GS) results on
  GA results

500th Generation (45000 simulations)
50th Generation (4500 simulations)
Required RMSmax for gradient-based search are
Dmax 0.12, 0.1, 0.08

• GS methods can get stuck in a local minimum
• GS methods reduce running time (CDM 145
  simulations)

13
Automating Transformations from Floating Point
to Fixed Point

• Existing fixed-point tools
• Support fixed-point simulation
• Convert floating-point code to raw fixed-point
  code
• Manually find optimum wordlength by trial and
  error
• Automating transformations
• Fully automate conversion and wordlength
  optimization process (Proposed)

Floating-Point Program
Wordlength-Optimized Fixed-Point Program
Code Conversion
Wordlength Optimization
14
Code Generation for Fixed-Point Program

• Adder function in MATLAB

Function c adder_fx(a, b) c 0 a fi (a,
1,32,16) b fi (b, 1,32,16) c fi (c,
1,32,16) c() a b
Function c adder(a, b) c 0 c a b
Determined by designers with trial and error
(a) Floating point program for adder
(b) Raw fixed-point program
Function c adder_fx(a, b, numtype) c 0 a
fi (a, numtype.a) b fi (b, numtype.b) c fi
(c, numtype.c) c() a b
WL
S
FWL
fi(a, S,WL,FWL) is a constructor function for a
fixed-point object in fixed-point toolbox S
Signed, WL Wordlength, FWL Fraction length
(c) Converted fixed-point program for
automating optimization (Proposed)
15
Automating Transformation Environment for
Wordlength Optimization
Input Data
Top Program
Floating-Point Program
Optimum Wordlength
Evaluation Program (Objectives)
Search Engine
Fixed-Point Program
Gradient-based or Genetic algorithm
Error Estimation
Complexity Estimation
Range Estimation

• Given floating-point program and options,
• auxiliary programs are automatically generated
• Given input data, optimum wordlength is searched

16
Demo of Released Software
17
Conclusion

• Search for optimum wordlength
• Gradient-based search reduces execution time with
  complexity-and-distortion measure method while
  solutions could be trapped in local optimum
• Genetic algorithm can find distortion vs.
  complexity tradeoff curve, but it requires longer
  execution time
• Automate transformations from floating-point
  programs to fixed-point programs
• Free software release is available at
• www.ece.utexas.edu/bevans/projects/wordlength/con
  verter/

18
End
Thank you!
19
Backup Slides
Backup Slides
20
Case Study- Receiver Gradient-Based Search
Integrate Dump
Demodulate
Search Method Gradient Measure Number of Simulations Complexity Estimate (LUT) Distortion (RMS)
Gradient Gradient Gradient Complete DM CDM CM - 66 65 195 164 40.65 43.65 41.95 - 0.083 0.085 0.081 -
Maximum distortion measured by bit error rate
(BER) is 0.1
21
Case Study - Receiver Genetic Algorithm
Population for one generation 90
25th Generation
50th Generation
100th Generation
200th Generation
22
Fixed-Point Data Format

• Integer wordlength (IWL)
• Number of bits assigned to integer representation
• Fractional wordlength (FWL)
• Number of bits assigned to fraction
• Wordlength (WL)

SystemC format www.systemc.org
p 3.14159(10) Floating Point
3.140625(10) 011.001001(2)
WL9 IWL3 FWL6 3.141479492(10)
011.00100100001110(2) WL16 IWL3
FWL13
23
Wordlength Optimization Constraints

• Distortion constraint

• Complexity constraint

Application-specific distortion d(w)
Application-specific distortion d(w)
Dmax
Cmax
Implementation Complexity c(w)
Implementation Complexity c(w)
Enforcing both constraints bounds the search to
a finite area region
24
Wordlength Optimization

• Wordlengths of signals (variables) in digital
  system as vector

• Multiple objective optimization

• Single objective optimization

25
Pareto Optimality

• Pareto optimality best that could be achieved
  without disadvantaging at least one group Allan
  Schick 1970
• Pareto optimal set is set of nondominated
  solutions
• E is dominated by C as all objectives for C are
  less than corresponding objectives for E
• Solutions A, B, C, D are nondominated (not
  dominated by any solution)
• Pareto front is boundary (tradeoff curve) that
  connects Pareto optimal set solutions

Pareto Front
I
A
G
Objective 2
H
B
E
C
F
D
Objective 1
26
Comparison of Proposed Methods
Gradient-based search Genetic algorithm
Type of Solution One point Family of points
Tradeoff Curve Found No Yes
Execution Time Short Long
Amount of Computation Low High
Parallelism Low High
27
Automatic Transformation Flow

• Code generation
• Parse floating-point program
• Generate a raw fixed-point program and auxiliary
  programs (top, objective, cost, etc.)
• Range estimation
• Estimate range to avoid overflow
  (Analytical/Simulation)
• Determine integer wordlength (IWL)
• Wordlength optimization
• Optimize wordlength according to given input, and
  error specification (Analytical/Simulation)
• Determine fractional wordlength (FWL)

Code Generation
Wordlength Optimization
Range Estimation
28
Code Generations
ltRun Code Generationgt
ltFloating-point Programgt