GSRC

1
GSRC

• bX update
• March 2003
• Aaron Ng, Marius Eriksen and Igor
  MarkovUniversity of Michigan

2

• Outline
• Motivation, issues in benchmarking
• bX in the picture
• Sample application Evaluation of tools
• Future focus
• Contact info, links

3

• Motivation, issues in benchmarking
• Evaluation
• independent reproduction of results and
  experiments
• explicit methods required
• minimum room for misinterpretation of results
• evaluation of algorithms across entire problem
  space
• conflicting and correlating optimization
  objectives
• separation of placement and routing tasks

4

• Motivation, issues in benchmarking (contd)
• Availability of results
• raw experimental results
• availability allows verification
• results provide insight into the performance of a
  tool

5

• Motivation, issues in benchmarking (contd)
• Standard formats
• meaningful comparison of results
• compatibility between tools and benchmarks
• correct interpretation of benchmarks

6

• bX in the picture
• Automation
• live repository
• support for execution of tools on benchmarks
• distributed network of computational hosts
• online reporting of results
• automatic updates when changes in dependencies
  occur

7

• bX in the picture (contd)
• Scripts and flows
• reproduction of results
• scripts and flows describe experiments
• scripts can be saved, shared and reused
• representation of entire problem space
• relationship between optimization objectives
• e.g. the effect of placement results on routing

8

• bX in the picture (contd)
• Standard formats
• interoperability between tools and benchmarks
• meaningful comparison of results

9

• Sample application Evaluation of tools
• Placers
• Capo
• randomized
• fixed die placer
• emphasis on routability
• tuned on proprietary Cadence benchmarks

10

• Sample application Evaluation of tools (contd)
• Placers (contd)
• Dragon
• randomized
• variable-die placer
• tuned on IBM-Place benchmarks

11

• Sample application Evaluation of tools (contd)
• Placers (contd)
• KraftWerk
• deterministic
• fixed-die placer
• results typically have cell overlaps
• additional legalization step by DOMINO

12

• Sample application Evaluation of tools (contd)
• Benchmarks
• PEKO
• artificial netlists
• designed to match statistical parameters of IBM
  netlists
• known optimal wirelength
• concern that they are not representative of
  industry circuits

13

• Sample application Evaluation of tools (contd)
• Benchmarks (contd)
• grids
• 4 fixed vertices, n2 1x1 movables
• tests placers on datapath-like circuits
• known optimal placement
• results are easily visualized for debugging

14

• Sample application Evaluation of tools (contd)
• Example flow

A script in bX serves as a template describing an
experiment, and can be saved and shared. Scripts
are instantiated by defining the individual
components of the script. Flows are instantiated
scripts. Flows can be re-executed to reproduce
results.
post-processing
placement
post-processor
placer
benchmark
evaluation
parameters
evaluator
15

• Sample application Evaluation of tools (contd)
• Example flow (contd)

Flow parameters Capo PEKO
(default) placement map
overlap/legality
wirelength
After completion, the results of the jobs will be
automatically posted online. In the case of the
placement job, the results include wirelength and
runtime.
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
parameters
evaluation
post-processor
parameters
evaluator
evaluator
16

• Sample application Evaluation of tools (contd)
• Example flow (contd)

Flow parameters Capo PEKO
(default) placement map
overlap/legality
wirelength
post-processing
placer
post-processor
benchmark
parameters
post-processor
evaluator
17

• Sample application Evaluation of tools (contd)
• Example flow (contd)

If we swapped Capo with Dragon Capo
PEKO (default)
placement map overlap/legality
wirelength
placer
benchmark
parameters
post-processor
evaluator
18

• Sample application Evaluation of tools (contd)
• Example flow (contd)

If we swapped Capo with Dragon Dragon
PEKO (default)
placement map overlap/legality
wirelength
placer
benchmark
parameters
post-processor
evaluator
19

• Sample application Evaluation of tools (contd)
• Example flow (contd)

If we swapped Capo with Dragon Dragon
PEKO (default)
placement map overlap/legality
wirelength
placer
benchmark
parameters
post-processor
evaluator
20

• Sample application Evaluation of tools (contd)
• Example flow (contd)

Capo PEKO (default)
placement map
overlap/legality wirelength
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
parameters
evaluation
post-processor
parameters
evaluator
evaluator
21

• Sample application Evaluation of tools (contd)
• Example flow (contd)

Modify the flow Capo PEKO
(default) placement map
overlap/legality
wirelength
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
parameters
evaluation
post-processor
parameters
evaluator
evaluator
22

• Sample application Evaluation of tools (contd)
• Example flow (contd)

Modify the flow Capo grid
(default) placement map
overlap/legality
wirelength
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
parameters
evaluation
post-processor
parameters
evaluator
evaluator
23

• Sample application Evaluation of tools (contd)
• Example flow (contd)

Modify the flow Capo grid
(default) grid graph
overlap/legality wirelength
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
parameters
evaluation
post-processor
parameters
evaluator
evaluator
24

• Sample application Evaluation of tools (contd)
• Example flow (contd)

Modify the flow Capo grid
(default) grid graph
overlap/legality wirelength
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
parameters
evaluation
post-processor
parameters
evaluator
evaluator
25

• Sample application Evaluation of tools (contd)
• Example flow (contd)

Swap Capo with Dragon Capo grid
(default) grid graph
overlap/legality
wirelength
placer
benchmark
parameters
post-processor
evaluator
26

• Sample application Evaluation of tools (contd)
• Example flow (contd)

Swap Capo with Dragon Dragon
grid (default) grid
graph overlap/legality
wirelength
placer
benchmark
parameters
post-processor
evaluator
27

• Sample application Evaluation of tools (contd)
• Example flow (contd)

Swap Capo with Dragon Dragon
grid (default) grid
graph overlap/legality
wirelength
placer
benchmark
parameters
post-processor
evaluator
28

• Sample application Evaluation of tools (contd)
• Example flow (contd)

Capo PEKO (default)
congestion map
overlap/legality wirelength
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
evaluation
parameters
parameters
evaluator
post-processor
evaluator
29

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) congestion map
overlap/legality
wirelength
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
evaluation
parameters
parameters
evaluator
post-processor
evaluator
30

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) congestion map
overlap/legality
wirelength
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
evaluation
parameters
parameters
evaluator
post-processor
evaluator
31

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) congestion map
overlap/legality
wirelength
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
evaluation
parameters
parameters
evaluator
post-processor
evaluator
legalization
legalizer
32

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) congestion map
overlap/legality
wirelength DOMINO
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
evaluation
parameters
parameters
evaluator
post-processor
evaluator
legalization
legalizer
legalizer
33

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) congestion map
overlap/legality
wirelength DOMINO
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
evaluation
parameters
parameters
evaluator
post-processor
evaluator
legalization
legalizer
legalizer
34

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) congestion map
overlap/legality
wirelength DOMINO
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
evaluation
parameters
parameters
evaluator
post-processor
evaluator
legalization
legalizer
legalizer
35

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) congestion map
overlap/legality
wirelength DOMINO
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
evaluation
parameters
parameters
evaluator
post-processor
evaluator
legalization
routing
legalizer
legalizer
router
router
36

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) congestion map
overlap/legality
wirelength DOMINO
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
evaluation
parameters
parameters
evaluator
post-processor
evaluator
legalization
routing
legalizer
legalizer
router
router
37

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) overlap/legality
wirelength DOMINO
post-processing
placement
placer
post-processor
placer
benchmark
benchmark
evaluation
parameters
parameters
evaluator
post-processor
evaluator
legalization
routing
legalizer
legalizer
router
router
38

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) overlap/legality
wirelength DOMINO
placement
evaluation
placer
placer
benchmark
evaluator1
benchmark
parameters
parameters
evaluator1
legalization
legalizer
legalizer
39

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) overlap/legality
wirelength
DOMINO
placement
evaluation
placer
placer
benchmark
evaluator1
benchmark
parameters
evaluation
parameters
evaluator1
evaluator2
legalization
legalizer
legalizer
40

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) overlap/legality
wirelength routability
DOMINO
placement
evaluation
placer
placer
benchmark
evaluator1
benchmark
parameters
evaluation
parameters
evaluator1
evaluator2
evaluator2
legalization
legalizer
legalizer
41

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) overlap/legality
wirelength routability
DOMINO
placement
evaluation
placer
placer
benchmark
evaluator1
benchmark
parameters
evaluation
parameters
evaluator1
evaluator2
evaluator2
legalization
evaluation
legalizer
legalizer
evaluator3
42

• Sample application Evaluation of tools (contd)
• Example flow (contd)

KraftWerk PEKO
(default) overlap/legality
wirelength routability
DOMINO timing analysis
placement
evaluation
placer
placer
benchmark
evaluator1
benchmark
parameters
evaluation
parameters
evaluator1
evaluator2
evaluator2
legalization
evaluation
legalizer
legalizer
evaluator3
evaluator3
43

• Future Focus
• Easy deployment
• downloadable bX distribution
• in the form of a binary or installation package

44

• Future Focus (contd)
• Interpretation of results
• multiple views and query support
• for example,
• show all results for solver S
• show the hardest benchmarks for solver S
• has the solution quality decreased for benchmark
  B,
• since the upload of the new version of solver
  S?

45

• Future Focus (contd)
• Type checking
• MIME-like affinity between solvers and benchmarks
• compatibility checks
• useful for performing queries on different
  families
• learning of new file types

46

• Future focus (contd)
• GSRC Bookshelf
• populate bX with implementations from Bookshelf
• still the same one-stop-shop
• except that it will be a live repository

47

• Future Focus (contd)
• OpenAccess
• method of communicating data between jobs
• provide interoperability between tools
• single design-through-manufacturing data model

48

• Contact info, links
• For more info or source code
• bx_at_umich.edu
• Feedback and comments are appreciated.
• OpenAccess www.openeda.org
• www.cadence.com/feature/open_access.html
• GSRC Bookshelf www.gigascale.org/bookshelf
• Thanks!