Modeling and Controlling the Incremental Software Test Process

1
Modeling and Controlling the Incremental Software
Test Process
Ray. A. DeCarlo School of Electrical and
Computer Engineering Purdue University, West
Lafayette, IN Aditya P. Mathur Department of
Computer Science
Scott Miller (CS) Department of Computer Science
Friday September 30, 2004. COMPSAC 2004. Hong
Kong.
2
The Problem

• What actions must the management take in order
  to adhere to process schedule?

3
Approaches

• 1. Use instinct and experience.

2. Use (1) supported by quantitative tools.
(a) Use simulation forward approach
(b) Use (a) plus feedback control inverse
approach.
4
Decision Support via Feedback
Actual Process
5
A Flow Model of Incremental Software
Development/Test
Test Specs
6
Workforce Allocation

• Workforce allocated to particular tasks
• Effort is split across all active tasks

7
State-Model Example Equations

• System State
• Progress
• Feature Coding (fc)
• Code Debugging (dr)
• Test Authoring (ta)
• Test Debugging (td)
• Regression Testing (rr)
• Defect Model
• Development
• Testing

8
Variable Productivity Equation

• Human Productivity
• Workload Dependent (Csikszentmihalyi,88)

• rb Base Work rate
• c Fractional size-dependent increase

• wc Current workload size
• wn Nominal workload size

9
The Productivity Eqns.

• Process Productivity (E.g. Feature Coding)
• Defect Introduction
• Defect Detection (Cangussu et al., 02)

10
Control Strategy

• Model Predictive Control

11
Model Predictive Control

• Select Cost Functional
• E.g.
• Q1,Q2 positive definite
• Calculate
• where Sxk, uk,kP ? xpk,kP

12
Initial Study

• Data Collection Questionnaire
• Unavailable Data Estimated by Mgmt.
• Many Linear Approximations
• Many subjective estimates
• Results follow

13
Expected Coding Progress
Features Rel. 3
Features Rel. 2
Features Rel. 1
14
Expected Test Authoring Progress
Tests Rel. 3
Tests Rel. 2
Tests Rel. 1
15
Expected Test Debugging Progress
Tests Rel. 3
Tests Rel. 2
Tests Rel. 1
16
Expected Regression Progress
Regression Tests Rel. 3
Regression Tests Rel. 2
Regression Tests Rel. 1
Scheduled
Inferred Actual
Simulated
17
Ongoing Study

• Data collection tool
• Objective parameter estimation
• Will reuse the tailored flow model
• Goal Assess predictive accuracy
• Goal Observe predictive accuracy over evolving
  process (i.e. training data growth)