TTCN

1
TTCN

• TTCN Testing and Test Control Notation
• Original meaning Tree and Tabular Combined
  Notation
• Original purpose Specification of abstract test
  suites for conformance testing of International
  Telecommunications Union protocols.
• Now promoted as a general purpose test language
  for distributed / communicating systems.
• Goal Provide a complete, internationally
  standardized, specification for test case
  behaviour and execution that is repeatable in any
  test lab.
• Predictable execution requires very precise
  semantic definitions.

2
History (1)

• Initial discussions on a standardized test
  language as part of a conformance test framework
  for communications protocols started in 1984.
• In 1994, the first version of the conformance
  test framework standard, ISO-9646, was published.
• TTCN-1 eventually was part 3 of this standard,
  published in 1995.
• Language assumed that the entire protocol testing
  methodology was being used.
• In 1997, TTCN-2 was issued.
• Most used in Europe only limited use in North
  America.

3
TTCN-2

• Tree and Tabular Combined Notation
• Tables describe data formats, signal
  specifications, etc.
• Trees used to describe test case behaviour
• Execution starts at the root of the tree
• Tree branches represent alternative execution
  paths.
• Implication no loops
• Alternatives explicitly specified for valid and
  invalid behaviour.

4
Implementation of TTCN Tests

• TTCN describes abstract tests that are completely
  platform independent
• The TTCN standard also provides operational
  semantics, which describe the exact execution
  behaviour of a test case
• Implementations are often in high-level language
  (C, Java)
• TTCN primitives are mapped to actual test
  environment.
• Implementations must obey the operational
  semantics

5
TTCN-2 Tables

• Example an abstract service primitive type
  definition

6
TTCN-2 Tables

• This table describes the data values to be used
  in one instance of a message of the type of the
  previous table, sent during a test case.

7
TTCN-2 Trees
expected forward progress in time
alternatives
8
TTCN-2 Test Suite Overview
9
TTCN-2 Test Suite Declarations
10
TTCN-2 Signal Declarations
message data format
11
TTCN-2 Table declarations
constraints
expected message data
12
TTCN-2 Test case
13
Implementing TTCN-2 tests
user interface
compiled TTCN test case
timer management
executor
encoder
decoder
form
extract
manually- written code
operating system service
PCO queues send/receive events
test equipment- specific
to system under test
14
Implementing TTCN-2 tests
user interface
compiled TTCN test case
timer management
executor
encoder
decoder
! A
? B
form
extract
011001.
110010.
PCO queues send/receive events
to system under test
15
History (2)

• Discussions after TTCN-2 centred on making the
  language more flexible and user-friendly.
• Standardization taken over by ETSI, the European
  Telecommunications Standards Institute.
• First version of TTCN-3 issued in 2001
• Radical change from the previous version

16
Credits

• Much of the following has been adapted from
• TTCN-3 An Introductory Overview
• Anthony Wiles, ETSI
• Conference presentation at TestCom
• About TTCN-3
• Tutorial available at www.etsi.org

17
What is TTCN-3?

• Testing and Test Control Notation Version 3
• Internationally standardized testing language
• Product of the European Telecommunications
  Standards Institute (ETSI)
• A programming language that has been used for
  more than 15 years in standardization as well as
  industry
• Specifically designed for black box testing and
  certification
• Constantly developed and maintained at ETSI by a
  team of leading testing experts from industry,
  institutes, and academia
• A testing technology that applies to a variety of
  application domains and types of testing
• Proven to work in very large and complex
  industrial tests, e.g., 3G network elements

18
TTCN-3 Standards

• Standards from ETSI some cross-listed as ITU
  standards
• ES 201 873-1 (Z.140) TTCN-3 Core Language
• ES 201 873-2 (Z.141) TTCN-3 Tabular
  Presentation Format (TFT)
• ES 201 873-3 (Z.142) TTCN-3 Graphical
  Presentation Format (GFT)
• ES 201 873-4 (Z.143) TTCN-3 Operational
  Semantics
• ES 201 873-5 TTCN-3 Runtime Interface (TRI)
• ES 201 873-6 TTCN-3 Control Interfaces (TCI)
• ES 201 873-7 Using ASN.1 with TTCN-3
• ES 201 873-8 The IDL to TTCN-3 Mapping
• Proposed Mappings to XML, C
• Currently Edition 3 of TTCN-3 (2007/8)

19
What makes TTCN-3 different

• From conventional programming or scripting
  languages?
• Rich type system including native list types and
  support for subtyping
• Embodies powerful build-in matching mechanism
• Snapshot semantics, i.e., well defined handling
  of port and timeout queues during their access
• Concept of verdicts and a verdict resolution
  mechanism
• Support for specification of concurrent test
  behaviour
• Support for timers
• Allows test configuration at run-time
• Tests focus only on implementation to be tested

20
What makes TTCN-3 different

• From a test tool or vendor proprietary testing
  language?
• Not tied to a particular application or its
  interface(s)
• Not tied to any specific test execution
  environment, compiler or operation system
• TTCN-3 as such is not executable and requires a
  compiler/interpreter, adapter as well as codec
  implementations

21
TTCN-3 test systems in a nutshell

• TTCN-3 specifies a test but a test system is
  needed for test execution
• TRI and TCI standards define test system
  architecture
• TTCN- 3 tools are required to support internal
  interfaces
• Allows reuse of test platforms with different
  tools but also for different SUTs
• A test system requires
• A TTCN-3 tool TTCN-3 compiler and execution
  environment
• A test platform for a specific device under test
• Note Tools come with default Test Control
  Logging

Test System Executor
TTCN-3 Test System
Test Control
Codecs
Logging
TCI
compiled
TTCN-3 Executable
TRI
SUT Adapter
Platform Adapter
System Under Test (SUT)
TCI TTCN-3 Control InterfaceTRI TTCN-3
Runtime Interface
22
An example adaptation A IPv6 test system
Test System Executor
TTCN-3 Test System
ParameterFile
IPv6 Codecs
ToolLogging
TCI
compiled IPv6 ATS
TTCN-3 Executable
TRI
Ethernet Adapter
Real-time Adapter
Open Source FreeBSD Router
23
Minimal Test Configuration
TTCN-3 Test System
TCI
Test Control
TTCN-3 Executable
Logging
MTC
Communication with IUT
Codecs
TRI
SUT Adapter
Platform Adapter
Communication with SUT
System Under Test (SUT)
Communication with IUT
Implementation Under Test

• All test behavior is executed on one (main) test
  component

24
Example Concurrent Test Configuration
TTCN-3 Test System
TCI
Test Control
TTCN-3 Executable
MTC
Logging
InternalCommunication
PTC
PTC
Codecs
TRI
SUT Adapter
Platform Adapter
System Under Test (SUT)
Implementation Under Test

• A test involves execution of many parallel test
  components
• Dynamic instantiation of components and
  communication links

25
Basic Concepts
26
Main Capabilities of TTCN-3

• Dynamic concurrent testing configurations
• Various communication mechanisms (synch and
  asynch)
• Data and signature templates with powerful
  matching mechanisms
• Specification of encoding information
• Display and user-defined attributes
• Test suite parameterization
• Test case control and selection mechanisms
• Assignment and handling of test verdicts
• Harmonized with ASN.1 and IDL (XML and C
  coming)
• Different presentation formats
• Well-defined syntax, static semantics and
  operational semantics

27
The Core Languageand Other Presentation Formats

• Core format is text based (most popular)
• TTCN-3 can be edited or viewed in other formats
• Tabular format (for TTCN-2 people)
• Graphical format (good for visual overview)
• Other standardized formats in the future?
• Proprietary formats possible

TTCN-3 Core Language
28
Example Core (Text) Format

• testcase TC_resolveEtsiWww() runs on DnsClient
• timer t_ack
• serverPort.send(m_dnsQuestion("www.etsi.org"))
• t_ack.start(1.0)
• alt
• serverPort.receive(mw_dnsAnswer("172.26.1.17"
  )) setverdict (pass)
• serverPort.receive // any other message
• setverdict(fail)
• t_ack.timeout
• setverdict(inconc)
• t_ack.stop

29
Example Graphical Format
testcase TC_resolveEtsiWww() runs on DnsClient
mtc
serverPort
DnsClient
DnsPort
timer t_ack
m_dnsQuestion("www.etsi.org")
t_ack
alt
mw_dnsAnswer("172.26.1.17")
pass
?
fail
t_ack
inconc
t_ack
30
Example Tabular Format
31
Use of TTCN-3 With Other Languages

• TTCN can be integrated with types systems of
  other languages
• Fully harmonized with ASN.1
• Harmonized with other languages
• IDL, XML, C/C

TTCN-3 Core Language
32
Major Elements of TTCN-3

• Built-in and user-defined generic data types
  (e.g., to define messages, service primitives,
  information elements, PDUs)
• Actual test data transmitted/received during
  testing
• Definition of the components and communication
  ports that are used to build various testing
  configurations
• Specification of the dynamic test system behaviour

TTCN-3 Core
33
Structure and Contents of a TTCN-3 Test Suite
34
TTCN-3 Modules

• Modules are the building blocks of all TTCN-3
  specifications
• A test suite is a module
• A module has a definitions part and a control
  part
• Modules can be parameterized
• Modules can import definitions from other modules
• Modules can have attributes

Module ()
35
Example of Module Structure
module MyModule (integer MyTSPar1) //
Definitions part control // Control
part with display ?ETSI format?
36
Module Definitions

• Definitions are global to the entire module
• Data Type definitions are based on the TTCN-3
  predefined types
• Templates and Signatures define the test data
• Ports and Components are used in Test
  Configurations
• Functions, Named Alts and Test Cases define
  behaviour

Signature Templates
37
Importing Definitions
// Importing single definitions import template
MyTemplate from MyModule // Importing
definitions of the same kind import all template
from MyModule // Importing all
definitions import all from MyModule //
inhibiting recursive import import template
MyTemplate from MyModule nonrecursive / To
avoid name clashes the imported definition is
prefixed with the identifier of the source module
/
38
External Definitions
// External constants may be defined external
const integer MyExternalConst // External
functions may be defined external function
MyFunction4() return integer
39
Groups of Definitions
// A nested group of function definitions group
MyTestStepLibrary group MyGroup1 function
MyTestStep11 function MyTestStep1n
group MyGroup2 function MyTestStep21
function MyTestStep2n
40
TTCN-3 Data Types
41
TTCN-3 Base Types
42
Example of Integer Type
/ integer sub-type RANGE definition / type
integer MyIntegerRange (1 .. 100) type integer
MyIntegerRange (0 .. infinity) / integer
sub-type LIST definition / type integer
MyIntegerList (1 , 2, 3, 4) / integer sub-type
as a mixed LIST and RANGE / type integer
MyIntegerListAndRange (1 ..5, 7, 9)
43
Example of a Structured Type
// record type definition type record
MyMessageType integer field1
optional, boolean field2 // record value
notation var MyMessageType MyMessage field1 1,
field2 true // Individual field
assignment MyMessage.field2 false
44
Arrays
/ Arrays are not considered to be types in
TTCN-3. Instead, they are specified at the point
of variable declaration / // integer array of 5
elements var integer MyArray5 // indexing
starts at zero MyArray0 1 // An initialiser
may be used to set array values MyArray
1,2,3,4,5
45
Defining Test Data
46
Uses of Data in TTCN-3 Test Specifications

• Data in TTCN-3 is used
• internally to the test component
• as information passed between test components
  within the test system
• as test data transmitted to and received from the
  System Under Test (SUT)
• The grouping mechanism can be used to collect
  data definitions of a similar kind e.g.,
• Service Primitives (ASP)
• Protocol Data Units (PDU)
• internal coordination messages

47
Constants (and Variables)
/ Constants are defined in the definitions part
of a module (and inside functions and test
components) / constant integer MyConstant
1 / Variables can only be declared in the
module control part (and inside functions and
test components) TTCN-3 does not support global
variables / var hexstring MyHexString
48
Example Message Definition
// Complex messages can be defined by
records type record MyMessageType MyMessage
field1 integer optional, field2 integer,
field3 integer, field4 integer, field5 boolean
optional, field6 hexstring optional, field7 Ano
therMessageType, field8 bitstring, field9 charst
ring (1 .. 20)
49
Data Templates

• Templates are special data values that have the
  following characteristics
• matching mechanisms
• they can be parameterized
• they can be modified to provide a simple form of
  inheritance
• Templates can be applied to simple types as well
  as complex types
• Are most often used to define the possible
  received values of messages and responses to
  remote procedure calls

50
Matching Mechanisms
51
Examples of Simple Templates
/ Template for a simple integer type / template
integer MyData1 template integer MyData2
(1, 2, 3) / This line in a test behaviour
description will match any incoming integer value
/ P1.receive(MyData1) / This line will match
on one of the incoming integer values 1, 2 or 3
/ P1.receive(MyData2)
52
Example Message Template
template MyMessageType MyMessage field1 1
ifpresent, field2 (1, 2, 3), field3 (0 ..
255), field4 not (1, 2, 3), field5 omit, field6
ifpresent, field7 ?, field8 '1011?1101'B, f
ield9 ?abc? ltgt ?xyz? length (20)
53
Parameterized Templates (1)
// Value parameterization template MyMessageType
MyMessage (integer MyPar) field1 MyPar, fie
ld2 true field3 110011B // Example use of
this template P1.send(MyMessage(1))
54
Parameterized Templates (2)
// Type parameterization type record
MyRecordType(MyMetaType) boolean
field1, MyMetaType field2 // field2 type not
defined var MyRecordType(integer)
MyRecordValue field1 true, field2
123 // field2 now of type integer
55
Modified Templates
// Writing template MyMessageType MyMessage1
field1 123, field2 true // and template
MyMessageType MyMessage2 modifies
MyMessage1 field2 false // Is the same as
writing template MyMessageType MyMessage3
field1 123, field2 false
56
Local Function Definitions
/ Functions can declare local constants,
variables and timers / function
MyFunction const hexstring MyConst
'FF00'H var integer MyVar 1 timer
MyTimer
57
Procedure Signatures
/ Procedure signatures are needed for
synchronous communication over procedure-based
ports (remote procedure calls) / signature
MyRemoteProc ( in integer Par1, out float
Par2, inout integer Par3 ) return
integer exception (ExceptType1, ExceptType2)
58
Signature Templates
signature MyRemoteProc ( in integer
Par1, out float Par2, inout integer
Par3 ) return integer exception (ExceptType1,
ExceptType2) / templates may be used to define
the actual parameters to a remote procedure call
/ template MyRemoteProc Mytemplate Par1
7, Par2 , Par3 MyConst
59
Specifying Elements of Test System Architecture
60
Test Components

• Test components are the entities on which test
  behaviour is executed (in parallel)
• Declarations may be made locally in the component
• A list of the ports used by the component must
  also be given
• Actual configurations are built dynamically in
  the test behaviour using operations such as
  create, connect etc.

Test Component
61
Kinds of Test Components

• There are three kinds of component
• MTC (Main Test Component)
• PTC (Parallel Test Component)
• a PTC that defines the Abstract Test System
  Interface

Abstract Test System
Real test system connected to an SUT
62
Communication Ports

• Test components communicate via communication
  ports
• A test port is modeled as an infinite FIFO queue
• Ports have direction (in, out, inout)
• There are three types of port
• message-based, procedure-based or mixed

PTC1
PTC2
63
Example Port Definition
// Definition of a message-based port type port
MyMessagePortType message in MsgType1,
MsgType2 out MsgType3 inout integer //
Definition of a procedure-based port type port
MyProcedurePortType procedure out
ProcedureSignature1, ProcedureSignature2
64
Example Component Definition
// Definition of test component type type
component MyComponentType // Local
declarations var integer MyVar timer
MyTimer // Ports used by the component MyMessag
ePortType P1, P2 MyProcedurePortType P3
65
Test System Interface
/ A user-defined component that defines the port
interface to the underlying real test system
/ type component MyTestSystemIntefaceType My
LowerPCOType LT MyUpperPCOType UT
66
Specifying Test System Behaviour
67
TTCN-3 Functions

• Functions are the building-blocks of test system
  behaviour
• Functions have local declarations and a program
  part
• Can be a 'pure' function doing some internal test
  system task or specify test behaviour using
  communication operations such as send and receive
• External functions are allowed
• There are some pre-defined functions (type
  conversion etc.)

Function ( . . . )
68
Pre-defined Functions
69
Function Definitions(1)
// Functions can be parameterised // in, out,
inout paramters. Default is in function
MyFunction (in integer MyPar) // Functions
can return values // Default is void function
MyFunction return integer
70
Function Definitions(2)
/ This function performs a generic task. It does
not use any communication operations / function
MyFunction / This function uses
communication operations consistent with the
ports offered by MyPTC / function MyFunction
runs on MyPTC
71
Test Case Definitions

• Test cases are a special kind of function
  executed in the control part of a module
• The interface part (runs on) references the MTC
  on which the test case will run
• The system part (system) references the test
  system interface component. Can be omitted if the
  test case only consists of an MTC
• The Behaviour part defines test case behaviour

Test Case ( . . . )
72
Example Test Case Definitions
// Parallel configuration testcase
MyTestCase() runs on MyMTCType system
MyTestSystemType // behaviour defined
here // Configuration consists only of an
MTC testcase MyTestCase()runs on MyMTCType
// behaviour defined here
73
TTCN-3 Program Statements and Operations
74
Kinds of Statements and Operations
75
Basic Program Statements
76
Example Program Statement
function MyFunction var integer x, j for
(j1 jlt10 jj1) if (MyGlobalFlag
true) xj2 log int2str(x) else
x j3
77
Operators (1)
78
Operators (2)
79
Pre-defined Functions
80
Creating, Building and Controlling Test
Configurations
81
Configuration Operations
82
Creating Components

• The master test component (MTC) is the only
  component that is automatically created when the
  test case is executed
• All parallel test components (PTCs) must be
  created explicitly using the create operation
• The create operation returns the unique component
  reference for the newly created component
• Components can be created at any point in a
  behaviour description

83
Example of Create
// Define a variable of type component type
component MyComponentType // Define a
variable of type component var MyComponentType
MyComponent // create the component MyComponent
MycomponentType.create
84
Connecting and Mapping Components
Test System
MyPTC
MyMTC
Abstract Test System Interface
Real Test System Interface
85
Example of Connect and Map
/ The connect operation is used to connect
components within the test system
/ connect(MyPTCP1, mtcP1) / The map
operation is used to connect components to the
abstract test system interface / map(MyPTCP2,
systemPCO)
86
Communication Between Test Components
87
Asynchronous Communication
88
Asynchronous Communication Operations
89
Examples of Asynchronous Communication Operations
MyPort.send(integer5) MyPort.receive(MyTemplate
(5, MyVar)) MyPort.receive(AltB) from
MyPartner MyPort.receive(MyType) -gt value
MyVar MyPort.receive(integer5) -gt sender
MyPeer
90
Synchronous Communication
91
Synchronous Communication Operations
92
Examples of Synchronous Communication Operations
// Calling a remote procedure MyPort.call
(MyProc, 30ms) to MyPeer MyPort.getreply(MyP
roc) -gt value MyResult MyPort.catch (MyProc,
Exception1) // Replying to a remote
procedure MyPort.reply (MyProc)
93
TTCN-3 Behaviour Trees
94
Behaviour Statements
95
Behaviour Statements
// Sequential statements are denoted by a
semi-colon function MyFunction1 MyFunction2 x
x1 MyPort.receive(myMessage) // Blocks
execution until message received MyPort.send(MyMe
ssage)
96
Alternative Behaviour

• Alternative behaviour is defined using the alt
  statement
• alt statements may only be the receiving
  communication operations and timer events
• receive, trigger, getcall, getreply, catch,
  check, timeout

S1 alt S3 S6 S2 alt
S4 S7 S5 S8
alt S9
S10

97
Schematic View of Alternatives

• Each alternative consists of
• a guard (possibly empty)
• an event, and
• an (optional) sequence of actions (including
  further alts)
• The TTCN-2 concept of snapshot semantics still
  applies

alt
guard1
event1
98
Example Alt Statement
alt MyPort.receive(MyMessage1)
MyPort.send(MyMessage4) alt
// Next level xgt1 MyPort.receive
(MyMessage2) xlt1 MyPort.receive(MyMessage3)

99
Named Alternatives

• Named alternatives (named alt) are descriptions
  of partial alternatives
• defined in the module definitions
• are named and may be parameterised
• Named alternatives are macro-expansions
• they are not functions
• Can be referenced from anywhere that it is legal
  to write a normal alt statement
• The expand statement allows a named alt to be
  explicitly expanded inside another set of
  alternatives

100
Example of Named Alt (1)
// Definition in module definitions named alt
MyNamedAlt L1.receive(MyMessage1) L1.rece
ive(MyMessage2) // Use of the named
alt testcase TC101() MyNamedAlt() // In-line
reference in test case alt L1.receive(MyMe
ssage3) expand MyNamedAlt() // Macro
expansion here
101
Example of Named Alt (2)
// Is the same as testcase TC101() alt L1.
receive(MyMessage1) L1.receive(MyMessage2)
alt L1.receive(MyMessage3)
L1.receive(MyMessage1) L1.receive(MyM
essage2)
102
Default Behaviour

• Default behaviour is defined using named
  alternatives
• Defaults must be explicitly activated and
  deactivated
• The expansion of defaults is applied to the end
  of the top-level set of alternatives in an alt
  statement
• and to all single receiving operations and
  timeouts

// Note that the single statement L1.receive(MyMes
sage1) // is equivalent to alt
L1.receive(MyMessage1)
103
Activate and Deactivate
named alt MyDefault L1.any.receive // Use
of the named alt testcase TC101() activate
(MyDefault) // Default applies from
here deactivate (MyDefault) // Default does
not apply from here
104
Interleaved Behaviour
/ The interleave statement specifies all
possible combinations of executing a given set of
alternative statements / interleave
PCO1.receive(MyMessageOne)
PCO1.receive(MyMessageTwo)
PCO1.receive(MyMessageThree)
105
Timers, Verdicts and SUT Operations
106
Additional Operations
107
Verdicts

• Verdicts pass, fail, inconc, none, error
• Each test component has its own local verdict
• can be written (set) and read (get)
• Global verdict returned by Test Case (read-only)

Verdict returned by the test case when it
terminates
MTC
PTC1
PTC2
108
Timer Operations
// Starting, stopping and reading a
timer MyTimer.start(20E-3)// Default unit is
seconds MyTimer.stop MyVar MyTimer.read //
Check if a timer is running if (MyTimer.running)
// timeout in an alternative
MyTimer.timeout
109
Executing and Controlling Test Cases
110
Module Control

• Module control is the dynamic part of a TTCN-3
  specification where test cases are executed
  (execute)
• Local declarations, such as variables and timers
  may be made in the control part
• Basic programming statements may be used to
  select and control the execution of the test
  cases

Control
111
Local Control Declarations
control const hexstring MyConst
FF00H var integer MyVar 1 timer
MyTimer
112
Executing Test Cases
module MyModule // Test cases are defined in
the module definitions testcase MyTestCase()
runs on MyMTC system MyTestSystem //
behaviour defined here control // and
executed in the control part execute
(MyTestCase())
113
Controlling Test Cases
// Test Cases return verdicts execute(MyTestCase()
) -gt value MyVerdict // Timeouts may be placed
on Test Cases execute(MyTestCase(),0.5) -gt value
MyVerdict // Test Cases can be used with
program statements while (xlt10)
xx1 execute(MyTestCase()) // Selection
of Test Cases if (MySelectionExpression)
execute(MyTestCase())
114
Specifying Additional Attributes
115
Defining Attributes

• Special attributes can be associated with most
  TTCN-3 language elements using the with statement
• encoding information (encode)
• standardized ASN.1
• application specific, proprietary
• display information (display)
• standardized tabular and MSC formats
• or proprietary
• user-defined (extension)
• proprietary

Language Element
116
Example Attributes
type record MyPDU integer
field1, ia5string field2, boolean
field3 with display ?ETSI Tabular
PDU // Tabular PDU encode ?BER? //
Encoding directive extension ?MyRule? // User
specific