Modeling and Validation

1
Modeling and Validation

• Victor R. Basili
• University of Maryland
• 27 September 1999

2
The Software Development Problem

• Problem Build and evolve no surprise software
  for ever-expanding set of customers and
  applications
• Need Gain intellectual control over software
  development
• Standard Solution
• build models to encapsulate knowledge
• validate and elaborate the models via systematic
  observation, experimentation, and analysis

3
The Nature of the Software Discipline

• The Software Discipline is a synthetic science
• Software not visible
• SE is basically first-article development, not
  manufacturing
• Expectation for rapid change

4
Models Needed to Answer Questions

• What is the appropriate process for developing a
  system with a specific set of properties in a
  specific environment?
• What kinds of architectures are appropriate for
  particular types of functional change?
• Requires Models of
• Software property tradeoffs
• Requirements and design decision rationales
• Decision making under uncertainty
• Human-computer tasks and collaborations
• Architectures
• Domains and product lines

5
Types of Software Models

• Product models
• Descriptions requirements, architecture, design
• Properties performance, reliability, usability,
  portability
• Process models
• Descriptions activity networks, simulations,
  strategies (waterfall, evolutionary, spiral,
  incremental)
• Properties cost, schedule, staffing levels,
  defect introduction and elimination
• Interested in relationships such as the cause
  effect relationship between process and products

6
Have Built Many Models

• Processes
• spiral incremental development, test
  techniques
• Products
• libraries for applications, generic
  architectures
• Resources
• local cost models, resource allocation models
• Changes and Defects
• defect prediction models, application
  specific defect types
• Quality
• reliability, ease of change models
• Product Progress
• actual vs. expected product size over time

7
Using Many Formalisms

• Mathematics
• functions, state machines, algebras, abstract
  data types
• Empirical models
• regression analysis, neural networks, pattern
  recognition
• Dynamic Models
• Petri nets, event simulators, continuous
  simulators
• Executable and non-executable notations
• programming languages, specification languages
  (Z, VDM)

8
Allowing Us to

• Characterize
• Describe/differentiate software processes and
  products
• Understand
• Explain associations/dependencies between
  processes and products
• Discover causal relationships
• Evaluate
• Assess the achievement of quality goals
• Assess the impact of technology on products
• Predict
• Estimate expected product quality and process
  resource consumption
• Motivate
• Describe whats needed to control and manage
  software

9
Laboratories Needed to Validate Models

• Models form the basis of the discipline
• But they are hypotheses embedded in the learning
  cycle of observation, model building, and
  experimentation
• Need laboratories for observation and
  experimentation
• Laboratories where practitioners build software
  systems
• - observe them working in their environment
• - identify the real problems

10
The Laboratory Environment

• Strengths
• Symbiotic relationship between research and
  practice
• Practitioner need to control/manipulate project
  solutions based on empirical evidence
• Researcher need to experiment/validate theories
• Difficulties
• short term needs (practice) vs. long term goals
  (research)
• accessing and collecting data
• comparative studies, repeatability
• Need to solve the problem of creating effective
  laboratories

11
Conclusion

• Question How do we build no surprise software?
  
• Requires a software discipline that
• builds models to encapsulate knowledge
• based upon observation
• learns by validating the models via analysis and
  experimentation
• Assumes
• collaboration between researcher and
  practitioner
• effective laboratories
• the kind of support that will allow this to
  happen