Wayne Zage and Dolores Zage

1
(No Transcript)
2
(No Transcript)
3
An Overview of Design Metrics Research in the SERC

• Wayne Zage and Dolores Zage
• Computer Science Department
• Ball State University

4
An Overview of Design Metrics Research in the SERC

• Wayne Zage and Dolores Zage
• Computer Science Department
• Ball State University

5
Outline

• Introduction
• Metric Definitions, Projects
• Research Results
• Design Metrics Analyzers
• Extending DM Technology
• Appendix

Click a button for more information
6
The Design Metrics Research Team

• Principal Investigators
• Wayne Zage Dolores Zage
• Research Associate
• J. Michael McGrew Jeff Zhang
• Industry Associates
• Scott Garner Jeff Stineburg
• Research Students
• James West Robin Shimp
• Rembert Parker Srirama Koneru Melissa May

7
Design Metrics Research at Ball State University
has been funded by

• National Science Foundation
• Software Engineering Research Center
• Motorola Corp.
• Nortel Technologies
• Telcordia Technologies
• Northrop Grumman Corp.
• Computer Sciences Corp.
• GTE Data Services
• Magnavox Electronic Systems Co.
• Harris Corp.
• Indiana Business Modernization and Tech. Corp.
• Ball State University

8
Long Term Goal
The goal of the design metrics research is to
support the design process of software
development by providing validated metrics that
can be used to highlight stress points in a
design and determine overall design quality.
9
Outline

• Introduction
• Metric Definitions, Projects
• Research Results
• Design Metrics Analyzers
• Extending DM Technology
• Appendix

Click a button for more information
10
Design
Metrics
through the
(Z)
Ages
11
D(G)
DC
D
D
i
Design
e
Metrics
through the
(Z)
Ages
12
D(G)
DC
D
D
i
Design
e
TSTADEIM
B
D
Metrics
through the
feedback
(Z)
Ages
13
Design Metrics Feedback Experiment

• We evaluated the performance of our
• design metrics as they were applied to ongoing
• software development in a controlled experiment
• at Northrop Grumman Corporation.

14
D(G)
DC
D
D
i
Design
e
TSTADEIM
B
D
Metrics
through the
feedback
(Z)
Ages
15
MERLIN

• A

M
E
t
R
f
r
e
i
o
j
c
r
u
s
L
v
e
e
m
n
g
a
t
o
I
a
d
o
c
e
y
N
l
s
o
f
t
w
a
r
e
16
System Improvement Strategies Determined by
Metrics Analysis
DESIGN METRICS
17
System Improvement Strategies Determined by
Metrics Analysis
DESIGN METRICS
18
D(G)
DC
D
D
VHDL metrics
i
Design
e
TSTADEIM
B
D
Metrics
through the
feedback
(Z)
Ages
OO metrics
SDL metrics
models
19
To capture all of the information in a design,
our metric D(G) consists of

• an external metric De
• an internal metric Di
• D(G) De Di

20
The External Metric De
De e1(inflows outflows) e2 (fan in fan
out)
2
2
1
3
2
De ( (223) (12) ) (12) 23
21
The External Metric De
De e1(inflows outflows) e2 (fan in fan
out)
2
2
1
3
2
De ( (223) (12) ) (12) 23
22
The External Metric De
De e1(inflows outflows) e2 (fan in fan
out)
2
2
1
3
2
De ( (223) (12) ) (12) 23
23
The External Metric De
De e1(inflows outflows) e2 (fan in fan
out)
2
2
1
3
2
De ( (223) (12) ) (12) 23
24
The External Metric De
De e1(inflows outflows) e2 (fan in fan
out)
2
2
1
3
2
De ( (223) (12) ) (12) 23
25
The Internal Metric Di

• Di w1(CC) w2(DSM) w3(I/O)
• where
• CC ( Central Calls ) are procedure or function
  invocations
• DSM ( Data Structure Manipulations ) are
  references to complex data types
• I/O ( Input/Output ) are external device accesses

26
Design Balance
De
Di
The Design Balance metric, DB, is defined as
average De
DB
average Di
27
Design Connectivity

• The Design Connectivity Metric, DC, is a measure
  of the "connectedness" of the set of modules
  under consideration.

28
The Design Connectivity Metric, DC, for a Modular
System M is
TOTAL - (ROOTS 2(CONROOTS)) 1

• DC(M)

TOTAL
where TOTAL is the total
number of modules being evaluated,
ROOTS is the number of disjoint subsets of
connected modules,
CONROOTS is the sum of the number of additional
modules
(excluding a "root" module)
with a fan-in of 0 in each of the
disjoint
subsets of connected modules.
29
Outline

• Introduction
• Metric Definitions, Projects
• Research Results
• Design Metrics Analyzers
• Extending DM Technology
• Appendix

Click a button for more information
30
(No Transcript)
31
Our First Major Analysis
V(G)
LOC
Di
De
D(G)
Mods Highlighted
11
11
11
12
12
Hltd.Mods w/Faults
44
56
89
50
100
Faults Found
37
51
94
53
97
False Positives
56
44
11
50
0
32
The design metrics have been computed on

• university-based projects
• CSCs STANFINS project
• systems from the US Army Research Lab
• Magnavoxs AFATDS project
• Harris ROCC project
• three Northrop Grumman projects
• PBX system from Telcordia Technologies
• telecommunications systems from Motorola

Results The design metrics have been excellent
predictors of error-prone modules.
33
Other Results
Modules w/faults
Fault-free Modules
314

• Number of mods

2070
355
54
De mean
The De mean for modules with faults was over six
times the De mean for the fault-free modules!
34
So far, our design metrics give a software
designer a good indication of where potential
trouble spots exist. The designer can take
actions such as

• look for alternatives to a particular part of a
  design
• assign difficult components to experienced
  developers
• provide extra testing effort for indicated stress
  points

35
Outline

• Introduction
• Metric Definitions, Projects
• Research Results
• Design Metrics Analyzers
• Extending DM Technology
• Appendix

Click a button for more information
36
The Need for Metrics Analyzers

• The metrics highlight error-prone modules, but
  the calculations are not consistent and are too
  slow when performed by hand.
• A 14,000 line Ada program required 420 hours to
  hand-calculate and analyze the metrics, but only
  seconds using the tool.

37
Design Metric Analyzer Reports

• Metric
• Stress-Point
• Content Information

38
The DMA Reports Can

• provide input for the software process
• design and code inspections
• maintenance
• documentation
• be used to inspect the process
• adherence to design guidelines
• compliance with standards

39
Outline

• Introduction
• Metric Definitions, Projects
• Research Results
• Design Metrics Analyzers
• Extending DM Technology
• Appendix

Click a button for more information
40
Extending Design Metrics Technology

• Motorola SDL project

41
Results of Pilot Study
The design metrics model was able to correctly
classify the difficulty level of 96 of the SDL
modules.
42
Metrics Analysis of the SDL Model Study Data

• Reviewed 56 states containing a total of 1476
  errors. Each state had at least 1 error and a
  maximum of 157 errors, as identified by a review
  of 581 DDTs.
• The three states with the largest error counts
  (157, 75, 61) also have the largest De metric
  values (80217, 5964, and 8645 respectively).

43
Extending Design Metrics Technology

• Motorola SDL project
• Nortel Technologies Predicting Performance
  Behavior

44
What We Propose To Do
Simulation
Metrics
Performance Model
45
Extending Design Metrics Technology

• Motorola SDL project
• Nortel Technologies Predicting Performance
  Behavior

• Telcordia Technologies Moving metrics
  technology into practice
• Raytheon VHDL and OO metrics

46
Design Metrics on Object-Oriented Industrial
Software
47
Study Data Characteristics

• C
• 9 classes
• 4 derived classes from one base class
• 67 class methods
• 8 free functions

48
Observations on Study Data Faults

• 22 faults were detected
• 7 of the class methods contained 20 faults
• 4 classes contained the 7 class methods with
  faults
• 2 of the free functions contained 2 faults

49
Mapping OO Constructs to Design Metric Primitives

• Design metrics at the class level
• Design metrics at the method and function level

50
Examples of Primitive Components
51
Examples of Primitive Components
52
Design Metrics at the Method Level

• CMDe (method inflows method outflows)
  (fan-infan-out)
• CMDi CentralCalls DataStructureManipulations
  I/O
• CMD(G) CMDe CMDi

53
Design Metrics at the Class Level

• CDe (class inflows class outflows) the sum
  of class methods CMDe
• CDi the sum of class methods CMDi
• CD(G) CDe CDi

54
Applying OO Design Metrics

• Applied to four C programs
• Three small object-oriented trial programs
• 500 LOC
• One object-oriented industrial software project
• 5000 LOC

55
OO System Metric Analysis
56
Future Research Direction

• Further investigation of the applicability of
  CDe, CDi, CMDe and CMDi as indicators of
  fault-prone components

57
Potential Benefits of Design Metrics Research for
the Software Product

• identification of design stress points
• early indication of project status
• limit design complexity
• modularization so that the resulting modules are
  both testable and maintainable
• analysis of the impact of changes on the rest of
  the system
• analysis of version or system builds

58
Potential Benefits of Design Metrics Research for
the Software Process

• identification of potential problems early in the
  life cycle
• selection of alternative designs
• modularization so that the resulting modules are
  both testable and maintainable
• allocation of testing resources
• review of design architecture as one considers
  rejuvenation alternatives

59
Outline

• Introduction
• Metric Definitions, Projects
• Research Results
• Design Metrics Analyzers
• Extending DM Technology
• Appendix

Click a button for more information
60

• DESIGN
• METRICS

DESIGN METRICS
Visit our homepage at http//www.cs.bsu.edu/metr
ics/
61
Design Metrics Analyzer

Design Metrics Analyzer
Design Metrics Analyzer
Design Metrics Analyzer
Design Metrics Analyzer
62
DMA Reports

• Metric
• Stress-Point
• Content Information

63
Metric Reports

• three composite design metrics, De , Di and D(G)
• three composite design metrics, De , Di and D(G)
  on packages
• thirteen metrics, both composite and primitive
  design metrics
• extra metrics, including LOC and counts of some
  syntactic constructs used
• design balance of the system

64
Stress-Point Reports

• module and package level
• summaries of the analysis of each of the design
  metrics
• De (de_stress.rpt)
• Di (di_stress.rpt)
• D(G) (dg_stress.rpt)
• statistical view
• names of highlighted modules and their file names

65
Content Information Reports

• overloaded
• module information
• files analyzed
• connectivity
• call structure

• call index
• variable table
• set analysis
• postscript Venn

66
Overloaded Report
overload.rpt

• contains a listing of overloaded module names and
  their parameters
• names are uniquely identified by appending three
  underscores and a constant to the module name
• informs developers of the number of modules with
  the same name and parentage
• helps with referencing module names from other
  reports

67
Module Information Report
unit_info.rpt

• textual summary of each module
• modules called and called by
• whether called modules were submitted to the tool
  (missing information)
• global variables accessed and/or modified
• imports of the module (with and use lists)

68
Files Analyzed Report
files_analyzed.rpt

• contains file names submitted to the tool and a
  physical line count for each
• provides a reference point for the user to check
  that the appropriate files were submitted to the
  DMA

69
Connectivity Report
connectivity.rpt

• input files can be partial systems
• this report groups modules according to their
  connectivity
• helpful in identifying subsystem divisions as
  well as the call structure for each

70
Connectivity Report Example
71
Call Structure Report
call_struct.rpt

• uses indented notation to signify subordinate
  modules
• helpful to developers, maintainers and testers

72
Example Call Structure
A
D
B
E
F
G
H
C
73
Call Structure Report Example
Subsystem 1 Section 1 Connected under root
module A page 1 reference 1 1 A 2 B 3
C
Subsystem 2 Section 1 Connected
under root module D page 1 reference 1 1 D 2
F 3 H 4 G
Subsystem 2 Section 2 Connected
under root module E page 1 reference 1 1 E 2
H
74
Call Index Report
call_index.rpt

• alphabetized module index for the call structure
  report
• provides the subsystem, section, page reference,
  and line number of each module listed in the call
  structure report

75
Call Index Report Example
Subsystem Section Page
Reference Line A 1 1 1 1 1
B 1 1 1 1 2
C 1 1 1 1 3
D 2 1 1 1 1
E 2 2 1 1 1
F 2 1 1 1 2
G 2 1 1 1 4
H 2 1 1 1 3 H 2
2 1 1 2
I 2 2 1 1 2
76
Variable Table Report
var_table.rpt

• provides a complete listing of variables and
  their types
• lists where each variable is declared
• variable access
• initialization and modification

77
Set Analysis Report
set_analysis.rpt

• listing of the module names that are identified
  as stress points by one, two or three of the
  design metrics

78

Set Analysis Report Example
79
Postscript Venn Report
venn.ps

• graphically shows the number in each intersection
  of modules highlighted by De, Di, and D(G) in
  the form of a Venn diagram

80
Postscript Venn Report Example
Highlighted Subprogram Analysis
De
Di
2
8
0
0
15
6
Di Highlighted 24
De Highlighted 23
D(G)
D(G) Highlighted 22
81
Other Reports

• generic.rpt
• rename.rpt