Acceptance Testing Strategy Guided by Evolutionary Operational Profiles

1
Acceptance Testing Strategy Guided by
Evolutionary Operational Profiles

• Authors Maria de Fatima Mattiello-Francisco
  fatima_at_iss.inpe.br
• Ana Maria Ambrosio
• National Institute for Space Research
• Edgar Toshiro Yano
• Aeronautical Institute of Technology

Industry Track ISSRE2007
2
Motivation
Acceptance Process
Service Profiles
Test Strategy
Results
The target is the Payload Embedded Software on
board X-Ray Astronomy Satellite
Principal Investigator
3
Acceptance Process
Service Profiles
Test Strategy
Results
Motivation
ECSS- European Cooperation for Space
Standardization
CDR Critical Design Review QR Qualification
Review AR Acceptance Review
4
Acceptance Process
Service Profiles
Test Strategy
Results
Motivation
Mission Integration Stages
The target software is integrated with other
mission's components in the three mission stages
Instrument
Subsystem
System
(Satellite Platform)
(Space Ground Segment)
(Payload)
During each stage the payload software will be an
encapsulated component being validated under the
different testers point of view
5
Acceptance Process
Service Profiles
Test Strategy
Results
Motivation
Testers involved in the Mission Integration Stages
Instrument
Subsystem
System
Payload
Satellite Platform
Space Ground Segment
U2 U3 U4 U5
U2 U3 U4 U5 U6 U7
U1 U2 U3 U4
U1- Payload Software Engineer (focus on software
requirement validation) U2 Payload Engineer
(focus on interfaces performance) U3 Principal
Investigator (Scientist focus on operational
modes) U4 Software Engineer (Regression Test
) U5 On Board Data Handling Computer Engineer
(focus on communication) U6 Mission System
Engineer U7 Ground Segment Engineer (Mission
Center Operation)
6
Service Profiles
Test Strategy
Results
Motivation
Acceptance Process
Operation problems can be anticipated by TESTERS
performing service-oriented test cases during the
acceptance testing
7
Service Profiles
Test Strategy
Results
Motivation
Acceptance Process
The proposed Evolutionary Operational Profile
Aggregates to Musas approach the service
operational profiles, adding the concept of
service and the usage of system architectural
elements
Such new approach aims at supporting a test
strategy for embedded space software acceptance
along the evolutionary integration stages
Denotes the growth in complexity and autonomy of
the environment with which the embedded software
interacts as far the integration stages evolves.
At each integration stage the environmental
facilities are incrementally substituted by the
real subsystems.
Software functionality does not change but
service operation does due to the test scripts
evolution
8
Service Profiles
Test Strategy
Results
Motivation
Acceptance Process
SERVICE PROFILES
Is a concept which depends on software behavior
in terms of operational mode and software
architecture
Services are performed within system modes by
component and their interfaces.
They are conceived taking into account the
following steps

• Identify software operational modes profile
• Identify software architectural profile
• List the SERVICES to be performed
• Establish relationship between element and
  services concerning USAGE INTENSITY
• Calculate the service operational profiles

9
Service Profiles
Test Strategy
Results
Motivation
Acceptance Process
Mode profile
Is defined as the probability of occurrence of
each operational mode in the system use.
A service might be provided by more than one
software operational mode and its occurrence
probabilities being different in each mode.
Software operations and services are performed
within operational modes and by architectural
elements.
10
Service Profiles
Test Strategy
Results
Motivation
Acceptance Process
Architectural Element profile
Case Study MIRAX Payload Software - SWPDC
Architecture
11
Service Profiles
Test Strategy
Results
Motivation
Acceptance Process
Architectural Element profile
The occurrence probability estimation depends on
high knowledge of the application and historical
data from previous experience in similar projects
12
Service Profiles
Test Strategy
Results
Motivation
Acceptance Process
Service Profiles Calculation
Service profile is defined as the occurrence
probability of each service in terms of using a
particular element.
MODE USAGE INTENSITY MATRIX
13
Test Strategy
Motivation
Acceptance Process
Service Profiles
Results
Test Strategy
Based on SERVICES PROFILES PSe, one selects the
number of TEST CASES Ntc provided by U testers
per service in order to match the previous total
time T allocated for each mission integration
stage
Ek
There is a set of services Si that exercise
better some elements Ek than other. Therefore,
priority on performing particular test cases is
recommended whenever reduction on testing
duration are imposed to the testers Uj
Uj
f ( S4,U2,E1) ???
Si
14
Test Strategy
Motivation
Acceptance Process
Service Profiles
Results
Instrument
Ntc provided by tester per service
U1 U2 U3
U4
A
101
636
46
88
870
Considering that the average time of 2 test cases
execution (including analyses) is 1 hour for
group A, this stage will require 435 hours to be
concluded. If the available time is 220 hours
(50) only, our test strategy approach reduces
respectively the number of test cases as showed
in group B
15
Results
Motivation
Acceptance Process
Service Profiles
Test Strategy
Faults detected by A and B
U1 U2
U3 U4
A
147 failures
B
74 failures
16
Results
Motivation
Acceptance Process
Service Profiles
Test Strategy
Questions???
Financial support to the QSEE Project Quality
in Space Application Embedded Software
http//www.cea.inpe.br/qsee/