Performance Evaluation: 30 years in review

1
Performance Evaluation 30 years in review

• Martin Reiser
• GMD German National Research Center for
  Information Technology
• reiser_at_gmd.de
• MMB99, 22.24.September 1999, Trier
• Presentation 23. September, 900

2
Very complex systems

• where are the limits?
• will it work?
• if so, how well?
• can it work better?
• what will I need tomorrow, in a month, year ?

but why bother anyway?
3
30 years?
4
Cutting the Gordian knot
Message Independence
Central Server
Kleinrock, 1964
Gaver, 1967 Buzen, 1971
5
Big bang BCMP
open class r
class change
i
i
G()
closed class s
FCFS
LCFS/PR
j
j
PS
IS
unlimited waiting room
queue-dependent rates
Baskett, Chandy, Muntz and Palacios, 1975
6
The PF solution is robust
station i, class r
Routing details dont matter, only mean number of
visits does! Distribution of service-times does
not matter, only mean values enter into the
solution!
Õ
Õ
n
i
,
r

P
(
n
)
C
(
n
)
W
i
i
,
r
i
r

long live the Central Server
7
The classical models

Delay analysis of data networks (Kleinrock 64)
Sharing the CPU, i.e. PS (Kleinrock et. al. 64 )
Time-sharing model (Scherr 67)
Central Server (Buzen 71)
Polling, token-ring (Konheim et. al. 74 )
Aloha, CSMA (Kleinrock, Lam, Tobagi 73 )
8
From trees
3274
370
3705
3274
3705
9
From window flow-control
10
From Poisson arrivals
call
11
Methods
12
Methods
13
Problems
14
Problems
15
PE specialists
work on PE problems cached in PE terms
build tools
develop (and solve) models of real systems
are integral parts of development teams
IS professionals
use PE methodology in their daily work
(planning, managing, tuning)
really?
16
Success stories
PE-based engineering and development
Capacity planning
Network management
Software performance engineering
Failures ?
17
PE-basierte FE

• Skilled PE specialist group
• is integrated in RD team
• is of sufficient size (5 10)
• is highly respected
• and contributes to design!

All important design decisions are based on PE
results and analysis
Early prototype support PE predictions
Case study IBM Token Ring
18
VLSI Design
Generally accepted specification process
Simulation data from specification
on several levels of detail
Automatic mask generation
Generation of test patterns
integrated process environment
19
PE in standards
Early attempts in SDLC/HDLC
Success story LAN standard IEEE 802
Success story ATM
fix before build
20
What is PE?
yes
Is it a science
some
Is it used by engineers?
little
Is it used in IS planning/tuning?
cottage industry
Is it a business?
Is it regular part of
not really
university curriculum?
Quo vadis ?
21
Stay where we are?
accepted in EE, especially communications
continue existence as niche science
22
SPE roadmap
Usage
Path-length
CASE me-
Rep.
estimation
scenario
thodology
process
editor
System
Capacity
structure
planning
editor
Workload data
Analytic system structure
Network capacity
model
planning
In users language
23
3 grand challenges
Understand self-similar traffic do we
need a new theory? se we merely
shifting loads?
Introduce analytical methods in
computer architecture (a gold mine S. Adve)
to enter into the main stream of the CASE process
and tools