Future Directions in Performance Evaluation Research

1
Future Directions in Performance Evaluation
Research

• Panelists
• Frederica Darema, NSF
• Albert Greenberg, Microsoft Research
• Adolfy Hoisie, Los Alamos National Lab
• Don Towsley, Umass-Amherst
• Organizer
• Evgenia Smirni, WilliamMary

2
Historical Overview

• Past versus future
• Overview of past -- looking at Sigmetrics
  proceedings
• Conference started in 1973
• Research topic shifts in different decades
• Theory versus practice
• Application areas

3
Early years 1970s

• Bimodal focus Theory -- Measurements
  (benchmarking)
• 1975 BCMP paper (JACM)
• Applications virtual memory, multiprogramming,
  working set, paging,instrumentation and
  measurements, hardware monitors, business systems
• Hardware IBM, CDC, PDP-11/70, VAX/VMS

4
Queuing Networks Era 1980s

• Clear focus on queuing networks most (now
  textbook) results
• 1980 Mean Value Analysis (Reiser/Lavenberg,
  JACM)
• 1983 Stochastic Petri Nets paper
  (Marsan/Conte/Balbo, Sigmetrics83)
• ApplicationsI/O, storage, DBs, distributed
  parallel systems, pipelining, networking
• protocols, multiprocessor systems, shared memory
  multoprocessors
• 1986 1st Joint Sigmetrics/Performance conference
• Position paper Insularity of
  Performance Evaluation (Ferrari, IEEE TSE)
• Hardware DEC, Cray IM, Cray X-MP, Cyber 205,
  hypercubes
• Applications are becoming more concrete, parallel
  and distributed topics dominate the late 80s

5
Systems Applications 1990s

• Clear focus on systems measurements, new
  architectures, parallel systems
• System support for performance of emerging
  systems and applications
• Applications scheduling, multimedia servers,
  traffic modeling, multi-level caching, Internet
  and
• Web servers, storage, TCP/ATM, databases
• Trends
• Early 90s parallel and distributed systems
• Late 90s networking
• Theoretical contributions declining

6
Networking Applications 2000s

• Applications moving from systems to networking
  applications
• 2000 Best systems paper award,
• 2006 Best paper in networking
• 2007 Best paper in systems
• Applications networking applications from 30 to
  50 of the conference papers
• Emerging applications energy aware, sensor
  networks, traffic engineering
• Increasing trend in theoretical contributions

7
Turning to the panelists
Frederica Darema (NSF) Research supported by
NSF Albert Greenberg (Microsoft
Research) Industry perspective Adolfy Hoisie,
Los Alamos National Lab Research supported by
DOE Don Towsley, Umass-Amherst Academic
perspective