Traditional Performance Measures

1
Traditional Performance Measures

• Capacity Reliability the probability that the
  system is outside the set of failures. The set of
  failures varies from applications to
  applications.
• Expected Capacity Survival Time the expected
  time for the capacity reliability to drop to a
  specific value

2
Traditional Performance Measures

• Capacity Reliability the probability that the
  system is outside the set of Computation
  Reliability R(s,t,T) is the probability that
  the system can start a task T at time t and
  successfully execute it ti the completion. The
  system state at time t is s, which is assumed to
  be a functional state.
• Computation availability which is the expected
  value of the system computation capacity at any
  given time

3
Example
Reconfigurable fault-tolerant system composed
4?
5?
8?
7?
3?
6?
2
2
0
1
0
1
Computing Realibility
The probability that the system is in state i at
time t
pfail ??i(t)
i?FAIL
4
Analysis

• Drawbacks Doesnt show you any form of mechanism
  which shows an interplay among the hardware,
  software and the application software
• Reliability typically focuses on either the
  hardware or software, independently of each other
  
• Availability is practically useless as a measure
  of the ability to meet execution deadlines.
• Throughput average measure ? conveys nothing
  about response time

5
Real-Time Performance Measures
6
Performability

• Improves traditional measures by
• Explicitly
• Formally accounting for the fact that the
  performance of a real-time computer should be
  tied to the consequent performance of the process
  that it controls.

7
Performability (cont.)

• The controlled process is defined as having
  several accomplishment levels (which are
  different levels of performance seen by the user)
• Every such level is associated with the execution
  of a certain set of control tasks.
• To accomplish each control task requires the
  real-time computer to run a set of control
  algorithms.

8
Performability is defined as the probability
that the computer system will allow each
accomplishment level to be met.

• More formally, if there are n accomplishment
  levels
• A1, A2,, An,
• The performability of the real-time system is
  given by the vector(P(A1), P(A2),,P(An) where
• P(Ai) is the probability that the computer
  functions in such a way as to have the controlled
  process as seen by the user is linked to the
  performance of the real-time systems

9
Thus, the quality of performance of the
controlled process as seen by the user is linked
to the performance of the real-time system
10
Hierarchical View of Performance

• Each view is more detailed than the one above it
• Each view is driven by the requirements of the
  one above it and receives input from the one
  below it
• By suitably defining the accomplishment levels,
  we can reduce performance to any one of the
  traditional performance measures
• E.g accomplishment level number of jobs
  processed per unit time ? throughput

11
Hierarchical View of Performance(cont.)

• View 0 specifies in terms of state variables of
  the controlled process (just what enables the
  user to distinguish one grade of performance from
  another)
• View 1 is more detailed in specifying the
  controlled process tasks that must be run
  (together with their associated timing
  constraints)

12
Hierarchical View of Performance (cont.)

• Each view is more detailed than the one above it
• View 2 is more detailed in that it specifies
  which algorithms must be run to meet each of the
  controlled-process task in View 1.
• View 3 considers the hardware structure, the OS
  and the application s/w attributes.

13
Hierarchical View of Performance
Users view of controlled process accomplishment
levels
View 0
Accomplishment of various controlled-process
tasks as a function of the operating environment
View 1
Capacity of the real-time computer to execute
specified control algorithms for various
controlled-process tasks
View 2
Hardware structure, operating system,
applications software
View 3
14
Case Study

• Landing portion of a flight. The aircraft has an
  automatic landing system, which allows it to land
  even in zero-visibility weather at suitably
  equipped airports. If the automatic-landing
  feature on the airport is not working when the
  landing phase begins, and if the destination
  airport has low-visibility weather, the aircraft
  is diverted to another airport. It is assumed
  that the alternative airport with sufficient
  visibility to permit a manual landing can always
  be found within a range of the aircraft. We
  assume that whenever a manual landing is
  possible, the automatic landing feature is
  dispensed with and the aircraft is landed
  manually. If the automatic landing feature fails
  during the automatic landing, there is a crash.
  Pilot errors are not accounted for.

15
Case Study (cont.)

• Accomplishment levels
• A0 Safe arrival at the designated destination
• A1 Diversion to another airport, but safe
  landing there
• A2 - Crash

16
Case Study (cont.)

• This can be translated into 2-tuple state
  description at View ) (a0,b0)
• a0

0 if the aircraft is not diverted
1 if the aircraft is diverted
0 if the aircraft does not crash
b0
1 if the aircraft crashes
17
Case Study (cont.)

• View 1- Operating environment is the weather at
  the airport. The control task that needs to run
  is the automatic-landing (AL) feature. The
  view-1 is represented in a 3-tuple (a1,b1, c1)
• a1

0 if the visibility of the airport is good at
designated airport
1 if the visibility of the airport is poor at
designated airport
0 if the AL feature is functional during the
landing phase
b1
1 if the AL feature fails before the landing
phase begins
2 if the AL feature fails during the landing
phase begins
18
Case Study (cont.)
0 if all the flight-critical mechanical parts
work correctly
c1
1 if there is a flight-critical mechanical
failure
The probability of being in each of these states
depends on the weather, on the state of the
mechanical components, and on the state of the
computer systems
Mapping Table
19
Case Study (cont.)
0 if the computer has sufficient resources to
run the AL job throughout the landing phase
a2
1 if the computer does not have sufficient
resources to run the AL job at any time during
the landing phase
2 if the computer has sufficient resources at
the beginning of the landing phase, but suffers
failures which make it impossible to run the AL
job sometime during the landing phase
The probability of being in each of these states
depends on the weather, on the state of the
mechanical components, and on the state of the
computer systems
Mapping Table
20
Map View-1 and View 2
0 if the visibility at the airport is good
?
1 otherwise
21
Calculating Performability

• Performability is a vector (P(A0), P(A1), P(A2))
  where, as before
• P(Ai) is the probability of the computer being
  able to function sufficiently well to ensure that
  accomplishment level Ai is attained.
• This is done by tracing through the mappings of
  the states in the 2 previous Tables by mapping
  between View 1 and View - 2

22
Calculating Performability (cont.)

• Example Accomplishment level A0 is attained when
  the system is in View-0 state (0,0), which
  happens whenever the system is in any of the set
  of
• View 1 states (0,0,0), (0,1,0), (0,2,0),
  (1,0,0)
• Which in turn happens whenever the states of the
  weather and the computer are
• (?,a2) ?(0,0), (0,1), (0,2), (1,0), and when
  c10

23
Calculating Performability (cont.)

• P(A0) Prob?0Probc10Prob?1Probb10Pro
  bc10
• Write P(A1) and P(A2)

24
Mapping of View-0 states to accomplishment levels
25
Mapping of View-1 states to View 0 states
26
Qualities of Performability
Rate at which each job will be initiated what
the hard real-time deadlines
Computer Architecture
27
Cost Functions Hard Deadlines
28
Cost Functions and Hard Deadlines

• Real-time tasks frequently have
• Hard deadlines the time by which they must
  finish executing if catastrophic failure of the
  controlled process is to be avoided.
• Cost functions of the response time. These
  functions are obtained by comparing the
  performability of a real-time system with zero
  response time to a system with given positive
  response time.

29
Cost Functions and Hard Deadlines (cont.)

• Formal definition
• The controlled process is meant to operate within
  a given state space. If it leaves this state
  space, it suffers failure.
• The hard deadline of any tasks is defined as the
  maximum computer response time that will still
  allow the process to be kept within the
  designated state space.
• The cost function of a particular task execution
  is given by P(?) P(0)
• where P(?) is the performability associated with
  a response time P?.

30
Cost Functions and Hard Deadlines (cont.)

• Example 2.8
• Consider a body of mass m, constrained to move
  only along one dimension and subject to impacts
  that change its velocity. Its state vector
  ?(x,v,a), consist of its current position x,
  velocity v, and acceleration a.
• The job of a real-time computer controlling m is
  to use thrusters that can exert a thrust of
  magnitude up to H in either a positive or
  negative direction to keep the body at a given
  ideal point for as much of the time as possible.

31
Derive the performability for this system

32
Cost Functions and Hard Deadlines (cont.)

• Example 2.8(cont.)
• If the body leaves a designated allowed state
  space SA, consisting of an interval of length 2b
  centered at the ideal point (which we can define
  as the origin), we say that the controlled
  process has failed.

b
-b
SA
33
Cost Functions and Hard Deadlines (cont.)

• What is the hard deadline associated with this
  process?
• Let us assume (for simplicity) that the
  controller delay (I.e. computer response time) is
  solely in recognizing that an impact has occurred
  and that the other reaction times are zero (that
  the thrusters can be turned on instantaneously).
• If the delay is too long, the controller may not
  be able to stop the body from moving out of the
  allowed space
• What is the cost function associated with this
  process?
• Let us define the accomplishment level as the
  energy expended by the system in getting the body
  back to the ideal operating point asap.
• The longer the controller takes to respond to
  the change in velocity, the more work ultimately
  which needs to be done in getting the body back
  to its ideal operating point.

34
Cost Functions and Hard Deadlines (cont.)

• Further observation
• The hard deadline and cost functions are
  functions of the current state of the controlled
  process, that is of the current position of the
  (x,v,a)
• In general the
• closer the body is to the boundary of the allowed
  state space or
• the faster it is moving away from the ideal point
• The shorter the hard deadline

35
Computing hard deadlines and cost functions at
several points
36

• The state of the process is defined by three
  tuples(x,v,a) where x,v and a are its current
  position, velocity and acceleration.
• Case 1 (0,0,0)
• The body is already at the ideal position, has
  zero velocity and is not accelerating. In such a
  case the controller has nothing to accomplish
  the body is already where it is supposed to be.
• Hard deadline infinity and the cost function is
  zero throughput

37

• The state of the process is defined by three
  tuples(x,v,a) where x,v and a are its current
  position, velocity and acceleration.
• Case 2 (x,0,0)
• The body is stationary at point x in the allowed
  state space, has zero velocity and is not
  accelerating. In such a case the controller has
  to bring it back to rest at the ideal point.
• No amount of controller delay will cause it to
  move out of the allowed state space
• Hard deadline infinity and the cost function is
  zero throughput (the energy expended is the same
  as that when it is zero)

38

• The state of the process is defined by three
  tuples(x,v,a) where x,v and a are its current
  position, velocity and acceleration.
• Case 3 (x,v,0)
• The body is in position x, has velocity v and is
  not accelerating
• The controller response time is ? that is it
  takes ? for the real-time computer to realize
  that an impact has occurred and must be responded
  to. By this time the body is in state (x v?, v,
  0)
• The controller must stop the movement of the body
  in the positive direction. This can be done by
  deploying a thrust of H in the negative direction
  which will need an acceleration of a -H/m

39

• Case 3 (x,v,0) cont.
• The body will come to a rest at position x1x
  v?v2/2a
• If x1 gt b, then the controlled process fails
• x v?v2/2a gt b ? ? gt (b-x v?v2/2a) / v
• That is the hard deadline is
• td(x,v,0) (b-x v?v2/2a) / v

40

• Case 3 (x,v,0) cont.
• The cost function
• The body is stopped at x1, which means that it
  has been decelerated over the interval x1-x. From
  this point, the body is accelerated towards the
  origin at maximal thrust until position x1/2

H
x1-x
x
x1
x1/2
41

• Case 3 (x,v,0) cont. The cost function
• The thrust (H)(I.e. Power) has to be maintained
  over a distance of dux1-xx1 2x1-x
• Energy distance (du) Thrust (H)
• The energy expenditure for response time zero is
• H(xv2/2a )

H
x1-x
x
x1
x1/2
42

• Case 3 (x,v,0) cont. The cost function
• The thrust (H)(I.e. Power) has to be maintained
  over a distance of dux1-xx1 2x1-x
• Energy distance (du) Thrust (H)
• The energy expenditure for response time zero is
• H(xv2/2a )

H
x1-x
x
x1
x1/2
43

• Case 3 (x,v,0) cont. The cost function
• The thrust (H)(I.e. Power) has to be maintained
  over a distance of dux1-xx1 2x1-x
• Energy distance (du) Thrust (H)
• The energy expenditure for response time zero is
• H(xv2/2a )
• Cost Function is
• Cx,v,o(u) H(2x1-x) - H(xv2/2a ) vuH

44
Discussion

• Performance Measures
• Means to convey the relative goodness of the
  system
• Efficient interfaces between the worlds of the
  user and the control engineer, and between the
  worlds of the control engineer and the computer
  engineer

45
Estimating Program Run Times
46
Estimating Program Run Times

• Very difficult
• Is a must ? due to the deadline
• Depends on the following factors
• Source Code
• Source code that is carefully tuned and
  optimized takes less time to execute
• Compiler
• maps the source-level code into a machine level
  program. This mapping is not unique the actual
  mapping will depend on the actual implementation
  of the particular compiler that is being used.
  The execution time will depend on the nature of
  the mapping

47
Estimating Program Run Times (cont.)

• Machine Architecture
• Machine architecture has an effect which is
  difficult to quantify. Interaction between
  processor(s), the memory and the I/O devices
• The number of registers in the CPU? memory access
  time
• Dynamic RAM ? Refresh ? priority over CPU

48
(No Transcript)
49
Estimating Program Run Times (cont.)

• Operating System (OS)
• Task scheduling and memory management both have a
  major impact on the execution time.
• Interrupt handling along with the machine
  architecture

50
These factors interact with each other and their
relationship accounting must be done in a precise
manner
51
Ideally, we would like a tool that would accept
as input the compiler, the source code, and a
description of the architecture ? it computes the
execution time of a code
52
We cant utilize experimental approach

• Because
• The actual number of potential input sets is very
  large and it would take too long to try them
• Individual programs do not run in isolation
  their run times are affected by the rest of the
  workload (including interrupts)
• Meaning that
• A more empirical approach is needed

53
Analysis of Source Code

• Lets us consider a simple stretch of code
• L1 a b c
• L2 b d e
• L3 d a f

Straight Line Code one entry point and one exit
point
Sequential control structure
54
Analysis of Source Code (cont.)

• L1 a b c The time needed to execute will
  depend on the code the compiler generates for it

L1.1 Get the address of c L1.2 Load c L1.3 Get
the address of b L1.4 Load b L1.5 Multiply L1.6
Store into a
55
Analysis of Source Code (cont.)

• The time needed depends on the machine
  architecture. If the machine is very simple, does
  not use pipelining and has only one I/O port to
  the memory, then the execution time is given by
  the sum

6
?Texec(L1.i)
i1
56
Analysis of Source Code (cont.)

• This assumes that
• We are implicitly assuming that variables b and c
  are not already in the CPU registers and have to
  be fetched from the cache or the main memory.
• Bound may be loose, because they are data
  dependent e.g multiplication depends on the
  numbers itself

57
Analysis of Source Code (cont.)

• Example ? loop

Only way is to acquire upper or lower bounds
omitted from Euclid real-time programming language
L4. while (P) L5. Q1 L6. Q2 L7. Q3 L8. end
while
58
Schematic of a timing estimation system
Produces compiled assembly language code marks
blocks of code to be analyzed
Preprocessor
Parser
Procedure Timer
Loop bounds
Time schema
Maintains a table of procedure s and their
execution time
Code Prediction
Parser analyzes the input source code
Architectural Analyzer
59
Schematic of a timing estimation system
Preprocessor
Parser
Procedure Timer
Loop bounds
Time schema
Loop bounds are obtained from various loops
Code Prediction does this using the code
generated by the pre-processor Using the
architectural analyzer
Code Prediction
Independent of the system depends on the
language computes the execution times of each
block
Architectural Analyzer
60
PERFORM - A Fast Simulator For Estimating Program
Execution Time

Alistair Dunlop and Tony Hey
Department Electronics and Computer Science
University of Southampton
Southampton, SO17 1BJ, U.K.
61
Simulation Program simulation is an alternative
method for predicting program execution time, but
has been generally dismissed on the grounds of
being too machine specific, extremely slow and
memory intensive. This is certainly a relevant
argument against using instruction-level
simulation and trace-driven simulation.
62
Event Driven Simulation The most recent form of
simulation, execution-driven simulation, lessens
these problems by interleaving the execution of
the input program and the simulation of the
architecture. A preprocessor parses either the
assembly language or source code of the program
requiring simulation. This code is modified to
extract information during program execution
hence the term execution-driven simulation. The
main advantage of this form of simulation is that
event generation and simulation are coupled so
that feedback can occur. This feedback can be
used to alter the ordering, timing and number of
subsequently generated events.
63
PERFORMA program slice is a fragment of a
program in which some statements are omitted that
are unnecessary to understand a certain property
of the program.
64
(No Transcript)
65
(No Transcript)