The State of Hybrid Model-Based Testing

1
The State of Hybrid Model-Based Testing

• Michiel van Osch
• 21-06-2005

2
Index

• Introduction
• Testing a Water Tank Controller
• Hybrid Model-Based Testing
• Conformance Relation
• Test-case generation
• Work in Progress
• Questions

3
Model-Based Testing
Discrete Model
Test-tool
Discrete System
4
Hybrid Systems
ABS
5
Hybrid Model-based Testing
Test-tool
Hybrid-Model
Hybrid System
6
Challenges

• Theory
• Develop Conformance Concepts
• Develop Test-case Generation Algorithms
• Prove Correctness and Applicability of Theory
• Practice
• Implement Tooling
• Exactly Model Relevant Aspects
• Connect Models, Tools and Systems

7
Index

• Introduction
• Testing a Water Tank Controller
• Hybrid Model-Based Testing
• Conformance Relation
• Test-case generation
• Work in Progress
• Questions

8
The water Tank Controller
set point
Tank Controller
messages
Water Tank
Heater
Temperature Sensor

• Tank Controller
• Discrete input Set point
• Discrete output communication
• Continuous input Temperature
• (from sensor)
• Continuous output Heat
• (from Heater)

9
Specification (1)

• Tank Controller
• Discrete input Set_setpoint(sp)
• Discrete output Print(warning)
• Continuous input ST
• Continuous output H

10
Specification (2)
ST
H
ST
H
input
output
input
output
Heater OFF
Heater ON
11
Test Setup
Tank Controller
Temperature Sensor
Heater
Specification
Test-tool
Implementation
12
Testing...
Test-tool
Specification
Implementation
Temp. Sens.
ST
Heat
H
Specification
Implementation
13
Testing
Set_setpoint(SP)
Set-point
Test-tool
Specification
Implementation
Specification
Implementation
14
Testing
Test-tool
Specification
Implementation
Specification
Implementation
15
Testing
fail
Print(warning)
error
Test-tool
Specification
Implementation
Specification
Implementation
16
Testing
fail
Test-tool
Specification
Implementation
Specification
Implementation
17
Index

• Introduction
• Testing a Water Tank Controller
• Hybrid Model-Based Testing
• Conformance Relation
• Test-case generation
• Work in Progress
• Questions

18
Conformance

• A hybrid implementation is input-output conform a
  hybrid specification if for all observable traces
  of the specification

The set of possible output actions and
trajectories, after the execution of a trace on
the implementation, is a subset of the set of
possible output actions or trajectories, after
the execution of a trace on the specification.
19
Some Decisions

• Defined on Hybrid I/O Automata
• Input Action Enabled
• Input Trajectory Enabled
• Non-zeno
• State Based possible output actions and
  trajectories in a certain state
• Input and output trajectories are treated
  together
• There is no quiescence observation (as of yet)

20
Conformance relation

• We Formally Define
• A hybrid trace ? ?0 a1 ?1 a2 as a sequence of
  discrete actions and trajectories of continuous
  variables (alternating)
• A after ? the set of states in which A can be
  after executing trace ?
• out(q) the set of output actions or trajectories
  possible in state q
• Out(Q) the set of output actions or trajectories
  possible in the set of states Q
• Then implementation HIOA i is hioco specification
  HIOA s if

i hioco s if ???traces(s) out(i after ?) ? out(s
after ?)
21
Examples
!a
!a
!c
Correct!
!b
!b
implementation
specification
Correct!
q
q
implementation
specification
22
Examples
!a
!a
!c
Incorrect!
!b
!b
implementation
specification
Incorrect!
q
q
implementation
specification
23
Test-Case generation Execution

• Input actions Select and stimulate
• Output actions Observe and validate
• Trajectories
• Select a time up to which to apply input
• Select an input trajectory to use for stimulation
• Stimulate and observe output variables
• Until the end of the trajectory, without
  observing a mismatch between observed and
  specified output trajectories, in this case
  continue
• Until an output action is observed, in this case
  validate the output action and if correct, then
  continue
• Until a mismatch between observed and specified
  output variables occurs
• As long as verdict fail is given, the verdict
  pass can be given

24
Test-case generation
fail
?Set_setpoint(SP)
SP gt ST
pass
fail
!Print(error)
!Print(warning)
pass
fail
25
Index

• Introduction
• Testing a Water Tank Controller
• Hybrid Model-Based Testing
• Conformance Relation
• Test-case generation
• Work in Progress
• Questions

26
Work in Progress (1)

• Does a continuous variable always have a value?
• State Changes During a Trajectory

27
Work in Progress (2)

• Quiescence
• Is quiescence a property on the discrete actions
  of the system or also on continuous behavior of
  the system?
• Notation

28
Current Future Work

• Further Develop these Concepts (Formalization,
  Theorems, Proofs)
• Make Concepts Workable in Practice
• Sampling of continuous behavior
• Selecting Realistic Input Trajectories
• Deal with Uncountable Many Possible Input and
  Output Trajectories and States
• Deal with Inaccuracy of Observed Output
• Implement Proof of Concept

29
Questions