Verification

1
Verification Validation

• Verification
• from Latin veritas meaning truth.
• Building the product right.
• Validation
• from Latin Valere meaning to be worth.
• Building the right product.

2
Tools for V V

• Must be able to support
• modelling
• simulation
• state-space search
• testing
• Must be scalable
• Must be able to operate on different platforms
• Unix or Linux
• Windows NT
• Windows 95,98

3
Example Telelogic Tau

• Modelling
• MSC(Message Sequence Chart)
• consists of entities and events occurring in time
  between those entities.
• SDL(Specification and Description Language)
• consists of a system defining the boundaries of
  the overall task.
• the system consists of components called blocks
  which are connected by channels.
• the blocks can consist of blocks and/or
  processes.
• the processes are the actors of the system and
  are series of actions, changes or functions that
  bring about a particular result.

4

• TTCN(Tree and Tabular Combined Notation)
• A tree contained in a table.
• Tree is composed of texts having different
  indentations. Texts having the same indentation
  have the same height in the tree.
• Each text is preceded by ? (incoming event) or
  !(outgoing event).
• State Space Coverage
• A graph that shows all the different combinations
  of inputs and outputs possible and their
  resulting transitions.
• It can be used to check for deadlocks and
  unspecified receptions.

5

• Simulation
• Tau contains a simulator that allows the
  interactive inspection and checking of the
  systems behavior.
• State-space search
• a bit-state or a random walk can be done with the
  validator in Tau. These will explore the state
  space of the SDL system to find the most possible
  run-time errors.
• Testing
• TTCN link
• Autolink

6
Case study Egg timer system

• An egg timer which boils an egg for the amount of
  time specified by the user.

7
SDLSystem Diagram

• A system defines a set of blocks and a set of
  channels through which interaction occurs between
  the environment and the system.
• Signals, signal lists and types are also defined.

8
SDL

• Specification and Description Language
• Initially built for communicating systems
• Divides the specification in
• Environment
• System(s)

The Environment Border
9
Into the blocks

• Blocks are composed of processes or of other
  blocks.
• Each block eventually contains processes.
• The processes communicate with the environment of
  the block via signal routes.

10
Signals and Signal lists

• Signals are the means through which processes
  interact.
• Signals to and from the environment are declared
  at the system level.
• Signal lists group signals that are used in the
  same grouping at several places.

11
SDL

• A Whole system

Block
System
Process
Procedure
12
Inside an SDL System
Channel (and channel name)
System name
Signals (Messages) Declarations
Signalist (set of messages)
Newtype definitions
13
SDL System Structure
Signallists are declared like signals, but
enclosed in parenthesis
Environment of the system
Channel name
System blocks
Signals are shown in square brackets, separated
by a comma
14
The EggTimer SDL Block
Connection points (system channel names)
Signals
routes
Procedure declaration
Process declarations
General declarations (signals, types, vars)
15
The Egg-Timer SDL Block Structure
Displaying sends to the environment and receives
from Timing etc...
Processes
At least 1 instance and at most 1 instance of
this process
Procedure Reference
16
SDL Process example
Initial state Initial transition
Declarations
State
Input
Procedure call
Timer Input (or timeout)
Output
If
Task
17
SDL Declaration example
To say youre declaring something
Identifier (instance name)
Instance type
Initialization
Timer declaration
Timer Identifier
18
SDL Transition example
FROM
WHEN
ACTIONS
SEND
TO
Local variable
Parameters to send
19
SDL Tasks
Setting the timer
Incrementing or attributing a value
Making a procedure call
Set(NOWltDurationgt, ltTimerNamegt) ltvariablegt
ltexpressiongt ltanother task
20
SDL Procedures(and operators)
Formal Parameters
Procedure Start
Param modifier name and type
If first seconds digit is 0 then go down,
else go left
Procedure return
Modifiers IN - Parameter is an Input
parameter OUT - Parameter is an Output
parameter IN/OUT - Parameter is both IN and OUT
21
SDL Message Handling

• Each process
• has one and only one message queue
• sends (by default) a message to the closest
  receiver (if the process itself is capable of
  receiving such message, it will send to itself)
• the default sending procedure can be modified by
• VIA
• TO

ltMessage_namegt VIA ltchannel_namegt
ltMessage_namegt TO ltprocess_idgt
22
SDL Message Handling

• This process queue has four possible messages
• IncMinute
• StartTimer
• ResetTimer
• OneSecond
• The first three are external messages where
  OneSecond is an internal message, indicating when
  the clock has decreased one second.
• There could be several other messages coming
• from other processes or other blocks

23
Simulation

• Once a complete SDL system has been designed, the
  Tau tool allows us to simulate the system i.e. to
  inspect and check its behavior.
• After having generated an executable simulator, a
  simulator user interface can be started. It looks
  like the following

24
(No Transcript)
25
Validation

• The validator is used to explore the state space
  to find errors or inconsistencies in the system.
• It can also be used to verify the system against
  requirements by running system level MSCs in the
  validator.
• An executable validator must be generated and the
  validator user interface can then be started.

26
(No Transcript)
27
Principles of Validation

• i) Design
• User-centred view
• scenarios
• use-cases
• MSCs
• functional, blackbox, providing a service (or
  NOT), end to end
• guided execution in SDL high-level design
• refined design, re-run validation scenarios

28
SDL Guided execution

• Using the Navigator in a SDL Validator tool.

Up 1
Next 1 Signal OneMinuteBtn
Next 2 Signal StartBtn
Next 2 Signal ResetStopBtn
...
Next 1 Tr. Start
Next i Tr. Start
By clicking on the Next, we go deriving a
scenario by following the state-space tree
Next 1 ... Input DisplayTime Sender
Timing1 Parameter(s)1,0,0
29
SDL Guided Execution(in MSC form)
30
Functional MSC for the previous EggTimer scenario
And executing this MSC in the Validator Tool, by
clicking on Verify MSC and selecting the proper
file, we obtain (as the system has been correctly
specified)
1 report
1 MSC Verification
MSC ValidatorTrace verified Depth 7
31
Principles of Validation

• ii) Test
• functional user interface tests in TTCN

S1
Test Name Event a Event b pass
c
a
a

b
b
S2
S3
Tree and Tabular Combined Notation

• TTCN standard for formal test case descriptions
• Facilitate sharing of tests
• Eliminates Ambiguities
• UNIX-based TTCN Workbench toolkit
• TTCN used for X.25, ISDN, CCS7, and many other
  protocols
• All PDU parameters specified
• All tests are formally assigned verdicts

32
Principles of Validation

• ii) Test (contd)
• basic syntax semantics of TTCN (TREE and TABLE
  Combined Notation)
• Test Case Dynamic Behaviour
• Test Case Name
• Group
• Purpose
• Default
• Behavior Constr. Ver
• Label Description Reference dict Remarks
• A!X
• A?Y
• A!J
• A?K Pass
• A?Z Fail Wrong
  Resp.
• A?Otherwise Inc. Test body
• not reached

33
Principles of Verification

• i) Design
• State-space search
• Process Chart
• Transition Chart
• State Space
• ?-transitions
• reduced state space

34
Principles of Verification

• i) Design (contd)
• Possible outcomes
• Unspecified reception
• queue overflow
• deadlock
• infinite livelock (no real progress)
• Random walk

35
Principles of Verification

• ii) Test
• Conformance Testing (TTCN)
• preamble
• test step
• verification sequence
• postamble

36
Principles of Verification
Structure of a Transition Test Case for
Conformance Testing
PREAMBLE Initialization to Preamble State
TEST BODY Application of Test Stimulus and
Confirmation of System Response
VERIFICATION Confirmation of Post-Transition
State
POSTAMBLE Reset to Stable State for Next Test
37
Principles of Verification
a/X
Represent
b/Y
S1
S3
S4
As a sequence of user actions and system
responses, starting in state 1 and finishing in
state 4. In TTCN Preamble(S1) !a ?X
!b ?Y
Verify(S4) PASS
FAIL
Inconclusive
38
Industrial Strength Tool Support

• Value added
• MSC coverage and test coverage.
• auto test generation.
• reverse engineering can be done to a certain
  extent.
• provides graphics editor(MSC SDL).
• provides editor for TTCN(ITEX).