Test case generation for Embedded Control Systems

1
Test case generation for Embedded Control Systems

• Dr. Qianchuan Zhao,
• Collin Spencer
• Prof. Bruce Krogh
• CMU
• Dec. 18, 2000

2
Method Framework
Embedded Control System Model
sliceMDL
Test Specification
Simplified System Model
Test Case Generator Engine
Test Cases
3
Global Search Technique
Embedded Control System Model
Test Specification
Global Search Engine
Input Region Computation
Test Cases
4
Theoretical Background
(based on evaluation of objective function)
5
A description of GA
SELECTION
6
GAOT toolbox

• A Genetic Algorithm Optimization Toolbox(GAOT)
  for Matlab 5
• Features
• Dealing with float point problems
• Mutation Operators uniform/non-uniform
  mutationboundary mutationmulti-non-uniform
  mutation
• Crossover Operators simple, arithmetic,heuristic
  crossover
• Selection Criteria probabilityRoulette
  wheelrankingtournament

7
Using GAOT toolbox

• ga command

x,endPop,bPop,traceInfoga(bounds,evalFN,evalOps
,startPop,opts, termFN,termOps,selectFN,selectOps,
xOverFNs,xOverOps,mutFNs,mutOps)
Example
x,bval,endPop,bestSols,tracega(bounds,'cw4tvg',
, initPop,-1,1,0,'myoptMaxGenTerm',100 0
0.1)
8
Mapping of TCG to optimization

• We map the Test Case Generation problem into an
  optimization problem which can be solved using
  the toolbox GAOT.
• Key observations in this work
• Use the actual simulink model
• Use special cost function
• Stop at first hit of the desired objective
  function value.

9
Use the actual simulink model
Simulate the model with the command
SIM T,X,Y1,...,Yn SIM('model',TIMESPAN,OPTIO
NS,UT) Example tout,xout,youtsim('whole4',Time_
Lengthdelta_t,opt)
10
Use special cost function

• To generate a specific output sequence, construct
  a non-positive objective function in a way such
  that it equals to zero if and only the desired
  output is the same as the real output. For
  example, the following is used to ask a output
  X,1,0,X,1.

F -((yout.signals.values(2)-1)2 (yout.signals
.values(3))2 (yout.signals.values(5)-1)2)
11
Use special cost function(contd)

• To guarantee coverage, guard conditions for
  switches are encoded by Boolean variables and
• For example, to ask both guards be toggled at
  least once(here thresholds are 0), we can use

F test(yout2) test(yout3)-2 function out
test(y) out max(ygt0) max(ylt0) 2 return
12
Stop at first hit

• Find optimal solution with known optimal
  objective function value (0).
• The search of optimal solution to our
  objective function will stop the first time the
  fitness value hits 0. So, essentially we are
  modeling the test case generation problem as a
  satisfiability problem and solve it with the help
  of global optimization tools.

13
Application
WS_INCONTROL
14
Application (contd)
15
Details
Intermediate inputs
WS_INCONTROL
16
Details
17
Test Problem
18
GA Example
Solving General optimization problem with
Genetic Algorithm Objective function F
-((yout.signals.values(2)-1)2(yout.signals.value
s(3))2(yout.signals.values(5)-1)2) Input
variable regions Based on the simulation data,
we determine the reasonable range of values of
inputs as bounds -1000,1000-4000,4000-10(5),
10(5) -10(5), 10(5)
19
Generated test case
20
Output
WS_INCONTROL
The output when applying generated test case
21
A full size example
22
Details
23
Generated test case
A test case
Feedback Data
Prefgen inputs
Ref Data
Compens inputs
24
Output
WS_INCONTROL
The output when applying generated test case
25
Evaluation of Algorithm
Experimental results on the algorithm
Note one iteration takes approx. 1 sec. on a
PIII machine
26
Coverage Example
A switch
A switch
27
Details
A switch
28
Generated test case
29
Output
The output when applying generated test case
30
Conclusions

• Status of our results
• We have derived a systematic test case
  generation procedure that can work on the full
  size models.

31
Future work

• Study the specification style for hybrid outputs.
• Study a method to incorporate the structure of
  model and empirical knowledge.
• Complete the Automation of the test
  specification procedure.
• Take care of fault detection and tracing.